-- Logs begin at Thu 2022-08-04 09:40:58 UTC, end at Thu 2022-08-04 10:41:52 UTC. -- Aug 04 10:11:15.835929 dv-cinder-826608-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 10:11:15.846937 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 10:11:15 2022] *** Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: nodename: dv-cinder-826608-8-iscsi Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: machine: x86_64 Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: clock source: unix Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: pcre jit disabled Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: detected number of CPU cores: 4 Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: current working directory: / Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: detected binary path: /usr/bin/uwsgi-core Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: your processes number limit is 47680 Aug 04 10:11:15.849001 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: your memory page size is 4096 bytes Aug 04 10:11:15.849684 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: detected max file descriptor number: 2048 Aug 04 10:11:15.849684 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: lock engine: pthread robust mutexes Aug 04 10:11:15.849684 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: thunder lock: enabled Aug 04 10:11:15.849684 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 10:11:15.849684 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 10:11:15.888285 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: Python main interpreter initialized at 0x55fd8acbf9c0 Aug 04 10:11:15.888285 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: python threads support enabled Aug 04 10:11:15.888285 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: your server socket listen backlog is limited to 100 connections Aug 04 10:11:15.888285 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: your mercy for graceful operations on workers is 90 seconds Aug 04 10:11:15.888604 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 10:11:15.888676 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: *** Operational MODE: preforking *** Aug 04 10:11:15.888947 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: *** uWSGI is running in multiple interpreter mode *** Aug 04 10:11:15.888947 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: spawned uWSGI master process (pid: 80155) Aug 04 10:11:15.889135 dv-cinder-826608-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 10:11:15.891985 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: spawned uWSGI worker 1 (pid: 80167, cores: 1) Aug 04 10:11:15.891985 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: spawned uWSGI worker 2 (pid: 80168, cores: 1) Aug 04 10:11:15.891985 dv-cinder-826608-8-iscsi devstack@keystone.service[80155]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 10:11:21.385379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:11:21.385379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes Aug 04 10:11:21.386147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:11:21.386147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes Aug 04 10:11:21.386547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:11:21.386547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes Aug 04 10:11:21.386651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:11:21.386651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes Aug 04 10:11:21.435658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 10:11:21.435846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 10:11:21.435931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 10:11:21.436048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 10:11:21.436129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] command line args: [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 10:11:21.436206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] command line args: [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 10:11:21.436320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 10:11:21.436399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 10:11:21.436500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ================================================================================ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 10:11:21.436578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ================================================================================ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 10:11:21.436756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.436833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_dir = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_dir = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_source = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_source = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] debug = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] debug = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.437738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.438964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_config_append = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_config_append = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_dir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_dir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.439982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.440977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.441858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.442985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.443057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.443465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.443542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.443663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] publish_errors = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.443736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] publish_errors = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.444908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] standard_threads = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] standard_threads = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.445995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_journal = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_journal = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_json = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_json = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_stderr = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_stderr = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_syslog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.446934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_syslog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.447078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.447152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:11:21.447333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.447409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.447571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.447648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.447830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.447904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.448913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.449892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.450857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.451829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.452843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.453954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.454987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.455943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.456848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.457983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.458920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.459007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.459154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.459236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.459410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.460960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.461886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.462839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.463942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464304 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.464957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.465979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.466923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.467850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.468845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.469935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.470926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471570 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.471880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.external = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.472885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.external = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.password = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.473872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.password = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.token = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.token = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.474958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.475949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.476957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.477944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.478954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.479938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.480988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.481944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.482900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.483868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.484844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.485970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.486106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.486316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.489989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.490936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.491931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.492936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.493864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.494982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.495935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.496763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.497983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.498905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.499902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.500066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.500576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.500576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.500885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.501829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.502928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.503976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.504940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.505907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.506079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.506158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.506839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.506913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.507936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.508059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.508149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.508492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.508867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.508948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.509860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.510868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.511850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.512959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.513970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.514951 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.515117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.515191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.515348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.515415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.515604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.516286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.516380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.516380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.516679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.516679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.517854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.518847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.519983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.520956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.521935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.522025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.522782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.522869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.523896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.524965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.525032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.525745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.525875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.526981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 10:11:21.527830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.527830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.528366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.528366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:11:21.528366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 10:11:21.556167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.556392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.556599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.556727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.556931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.557161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.564996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.565478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.595834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.596089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.596216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.596398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.648794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.649098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.699422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.699634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.700029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.700269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.764816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.765959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.766083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.766215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.766215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.789438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.789757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.789983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.790981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.791109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.791109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.791344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.791465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.802258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.802783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.815615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.815848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.816915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.817046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.817173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.817299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.817299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.817547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:11:21.835006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.835279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:11:21.835468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.835600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.835747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:11:21.835885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:11:21.836039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:11:21.836404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.836940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.837069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.837195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.846529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.846801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.846939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.847071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.847197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.847336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.847464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.847691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.854459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.855088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.859207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.859485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.859763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.859898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.860044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.860978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.861886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.862024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.862201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.862466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.886731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.888223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.888631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.888910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.889732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.890014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.893953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.894362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.894645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.895511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.895789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.896100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.902042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.902328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.903619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.903897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.906454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.906739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.907019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.907307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.907589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.907881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.908049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.908177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.908335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.908458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.908617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.908905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.909193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.909486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.909758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.910036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.924473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.924747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.926241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.926518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.929849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.931062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.933038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.933333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.934242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.934538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.937774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.938897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.941142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.942116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.944902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.945210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.945493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.945776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.945951 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.946080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.946251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.946378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.946536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.946820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.947109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.947401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.962366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.962648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.962919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.963192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.963317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.963609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.963858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.964155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.969991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.970967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.971536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.971809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.972548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.972823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.974647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.974921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.975267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.975577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.975703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.975854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.976000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.976234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.976515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.976789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.987194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.987470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.987743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:21.988307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.988950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.989203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.989488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:21.989766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.004866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.007824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.008475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.010826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.011399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.013782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.013915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.015424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.015665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.015945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.016297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.017827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.018112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.018393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.022359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.023989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.024872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.026488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.026883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.027171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.027454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.028316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.029434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.029722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.030005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.030279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.035799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.036122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.036402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.038323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.038613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.038893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.043373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.043645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.043927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.044224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.044491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.044766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.045293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.045576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.045966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:11:22.046235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.046843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.047861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.048276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.048550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.048820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.049081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.049351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.049617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:11:22.081411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55fd8acbf9c0 pid: 80167 (default app) Aug 04 10:11:22.084788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafae55-71d2-42fe-b925-5f50dd747ebc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:22.084959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55fd8acbf9c0 pid: 80168 (default app) Aug 04 10:11:22.085061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafae55-71d2-42fe-b925-5f50dd747ebc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:22.085335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafae55-71d2-42fe-b925-5f50dd747ebc None None] PATH_INFO: `/v3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:22.089622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 791 bytes} [Thu Aug 4 10:11:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:29.700278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:29.700843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:29.700843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:29.705259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1/2] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:29.710663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:29.711097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:29.711219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:29.782065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: INFO dbcounter [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] Registered counter for database keystone Aug 04 10:11:29.801058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG oslo_db.sqlalchemy.engines [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 10:11:29.807316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writer thread running {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 10:11:29.989019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 10:11:29.989347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 10:11:30.043594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:30.045469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:30.176368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 2/3] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 10:11:30.313683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:30.383644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: INFO dbcounter [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] Registered counter for database keystone Aug 04 10:11:30.404347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG oslo_db.sqlalchemy.engines [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 10:11:30.410418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writer thread running {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 10:11:30.526826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949d35-a32e-46b1-a8e8-25ec74b2deec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:30.531048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:30.531666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:30.531944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:30.532238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:30.540731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:30.591764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 10:11:30.592124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e574e11-5e48-4ccb-8291-c4b43c08f5ab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:30.601932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 2/4] 10.222.0.72 () {58 vars in 1324 bytes} [Thu Aug 4 10:11:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:11:30.607858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:30.622597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7c231-51a6-44d0-9496-6726a89b619a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:30.626934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:30.628072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:30.628356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:30.628626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] PATH_INFO: `/v3/projects/8d75901b5a184df7b4b0c986ca5abc96` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:30.630124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] RBAC: Authorizing `identity:get_project(project_id=8d75901b5a184df7b4b0c986ca5abc96)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:30.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 10:11:30.680506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d60101-08be-4c44-8d75-e8597355e1f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:30.685022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 3/5] 10.222.0.72 () {56 vars in 1207 bytes} [Thu Aug 4 10:11:30 2022] GET /identity/v3/projects/8d75901b5a184df7b4b0c986ca5abc96 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:32.651993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a4c4-eb6a-409c-a2f7-3fa76697abe6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:32.652502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a4c4-eb6a-409c-a2f7-3fa76697abe6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:32.652586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a4c4-eb6a-409c-a2f7-3fa76697abe6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:32.656649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 3/6] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:32.662004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:32.662461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:32.662553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:32.682456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:32.683981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:32.691652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 4/7] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 10:11:32.714174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54a4c4-eb6a-409c-a2f7-3fa76697abe6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:32.726725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54a4c4-eb6a-409c-a2f7-3fa76697abe6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:32.730937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:32.731530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:32.731813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:32.732118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:32.739690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:32.741524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a102c62-9839-4770-9af5-938474d59004 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:32.750133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 4/8] 10.222.0.72 () {58 vars in 1324 bytes} [Thu Aug 4 10:11:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:11:32.755228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:32.764564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd4bbd-ee58-4a01-8c92-edef9a089f84 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:32.768875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:32.769496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:32.769781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:32.770054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:32.771408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:32.773851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca575b1-753d-4c95-a587-5a3d449f894a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:32.775374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 5/9] 10.222.0.72 () {56 vars in 1198 bytes} [Thu Aug 4 10:11:32 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:34.820456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a28e3-4e0e-4969-99c0-2ea2af405be9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:34.820993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a28e3-4e0e-4969-99c0-2ea2af405be9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:34.821088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a28e3-4e0e-4969-99c0-2ea2af405be9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:34.822371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 5/10] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:34.827466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:34.827894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:34.828159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:34.847428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:34.848632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:34.909381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911f6-53bd-4bbd-a0ea-f2ec389588d6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:34.909669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911f6-53bd-4bbd-a0ea-f2ec389588d6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:34.909945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911f6-53bd-4bbd-a0ea-f2ec389588d6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:34.911267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 6/11] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:34.916460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:34.916746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:34.917021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:34.928247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 6/12] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:34.950689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:34.958790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 10:11:34.959786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 10:11:34.973824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.004740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:35.006838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:35.052099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733486f7-29af-4a01-aae1-2a49df72a687 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.085830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.086413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.086689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.086960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.097659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:35.104991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 7/13] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:35.106653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:35.107705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:35.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 7/14] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:11:35.113688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.126113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.206604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.212866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df803b-1e91-48cd-9c74-2a1aee677c2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.215173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.226251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984d3e-d5a4-4218-97b7-ff03977ff9c5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.252815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.253681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.253961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.254236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.255272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:35.255579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.257581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.264291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.264978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.265259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.265528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.269129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 8/15] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:35.274691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.332118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.333734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.370558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f8449-bc55-44ec-97c0-ad5229863824 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.404098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.404950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.405230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.405503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.407032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.408495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.410078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 9/16] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:35.414858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.415368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.434768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 8/17] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:11:35.440197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.451333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.523468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cffe9-896f-47ac-a310-604e3c954c46 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.559214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.559868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.560179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.560457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.561681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.563167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.564819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 10/18] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:35.571337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.653560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f4823-4702-452b-bf31-be86da9e40b5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.684265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480dcd1-93cb-4f42-900b-06ea99d7b9e0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.685841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.687379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.688102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.688541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.688707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.699142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.700642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:11:35.715293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:35.716231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:35.717014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:11:35.717014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:11:35.717014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:11:35.717014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:35.717014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 9/19] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:11:35.718195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.718883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.719168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.719441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.720256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.domain.id=default, user.id=bca06f724a4f42d0afd72d25da1e0911) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:35.720566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.721985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bae68e-6d96-4e34-9ff7-d16f7940875e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.723181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:35.731379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 11/20] 10.222.0.72 () {56 vars in 1322 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/role_assignments?user.id=bca06f724a4f42d0afd72d25da1e0911&scope.domain.id=default&role.id=abf8a86280c54a0798604e6c88e690cf => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:35.759213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:35.828452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8378fe-2043-494b-b94f-2845624a4286 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:35.863612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:35.864270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:35.864648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:35.864920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:35.866246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:35.866545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:35.867945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c93de-9cb0-4f59-a1f4-0d82340c7566 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:35.886391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 10/21] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:37.854146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed12b-30e0-4586-bf2f-6e195f46c9ee None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:37.855154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed12b-30e0-4586-bf2f-6e195f46c9ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:37.855659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed12b-30e0-4586-bf2f-6e195f46c9ee None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:37.857250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 12/22] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:37.862878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:37.863770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:37.864304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:37.883523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:37.885090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:37.955223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 11/23] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:37.999089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed12b-30e0-4586-bf2f-6e195f46c9ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.112500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed12b-30e0-4586-bf2f-6e195f46c9ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.148732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.149738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.150240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.150713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.163507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:38.172566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:38.173397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:38.174153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:38.176436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 13/24] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:11:38.185030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.210642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:38.301492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201aed0-8a16-43ae-9f90-0688fefa9a1e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.337315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.337952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.338226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.338502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.339228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:38.339534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:38.340969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2c710-996e-43cb-b586-dcb79d267341 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:38.353609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 12/25] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:38.359197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.366255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be078-bfb6-4457-9c9e-c161279345ef None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.366728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be078-bfb6-4457-9c9e-c161279345ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.367145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be078-bfb6-4457-9c9e-c161279345ef None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.368648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 13/26] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:38.378013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.378493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.379000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.388535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:38.416979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:38.418578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:38.473048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12a963-82f3-4026-b521-e7881b8e14f0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.512104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.513044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.513725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.514191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.515743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:38.517512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:38.519376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 14/27] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:38 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:38.526520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.542704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 14/28] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:38.579280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.616027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:38.637100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:38.641849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33fab-d6ac-4219-87e3-1aea4832b71a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.683641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.684582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.685164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.685606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.687238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:38.688960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:38.691175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 15/29] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:38.702443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:38.779746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7204cc-1e0e-42ef-8d57-7dd31ee5c3ed None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.821989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.822680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.822966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.823239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.824078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:38.825506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f52d37-2543-46ff-b738-b4f26fead2ac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:38.915764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a369-d4fd-4c8e-987b-1b165501e94d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:38.989616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:38.990843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:38.991134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:38.991417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] PATH_INFO: `/v3/domains/default/users/bca06f724a4f42d0afd72d25da1e0911/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:38.997511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bca06f724a4f42d0afd72d25da1e0911, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:38.998973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70813c91-80e8-42b8-ab0f-bdfb5d720d2d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:39.029113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 15/30] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Aug 4 10:11:38 2022] POST /identity/v3/roles => generated 219 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:11:39.030672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de34e2-d22d-4e2c-a875-291c62c4a27a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.031591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de34e2-d22d-4e2c-a875-291c62c4a27a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.031928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de34e2-d22d-4e2c-a875-291c62c4a27a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.033660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 16/31] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:39.041927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.042318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.042603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.084478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:39.084478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:39.207708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 17/32] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:39.231999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:39.250769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:39.307372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 16/33] 10.222.0.72 () {58 vars in 1361 bytes} [Thu Aug 4 10:11:38 2022] PUT /identity/v3/domains/default/users/bca06f724a4f42d0afd72d25da1e0911/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:11:39.364147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b0c0-0a43-4630-8c20-54cf63f841f1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.364394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b0c0-0a43-4630-8c20-54cf63f841f1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.364763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b0c0-0a43-4630-8c20-54cf63f841f1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.367640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091d05a-0e26-4694-9c2d-9bc4fd22fc91 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:39.369649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 17/34] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:39.384182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.384606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.384992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.414751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:39.415673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.416132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.416586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.432506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:39.433325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:39.434702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:39.435173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:11:39.467279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:39.468124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:39.468914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:11:39.468914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:11:39.468914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:11:39.468914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:39.469819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 18/35] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:39.477675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:39.478057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:39.563022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 18/36] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:39.592875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:39.616278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:39.654752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3c533-b410-4580-9a23-44960f1edfed None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:39.713329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:39.714379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.714862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.715332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.716263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:39.716785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:39.718449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:39.732271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 19/37] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:39.741453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:39.774723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59affc4-65a1-465a-826f-cd28bef4286a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:39.814898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:39.815524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.815808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.816110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.816849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:39.818959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef3d5e-9452-46bc-9b80-0d2f6c260392 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:39.852096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 19/38] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Aug 4 10:11:39 2022] POST /identity/v3/roles => generated 225 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:39.853367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7aea-b436-44ab-9d56-604744212c16 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.853657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7aea-b436-44ab-9d56-604744212c16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.853936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7aea-b436-44ab-9d56-604744212c16 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.855319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 20/39] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:39.864051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.864682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.866393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.891968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:39.894552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:39.906155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:39.912815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a001-c98e-4d85-ba84-4bca3942cbaa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:39.938383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:39.963178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:39.964695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:39.965230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:39.966264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:39.967388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:39.969964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f81c7-6664-447f-988d-5edc7a9ea569 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:39.983425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 21/40] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:40.008009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 20/41] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:11:39 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:40.014950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:40.019835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b5c22-61ec-4794-97b5-60fde553c36b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:40.019835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b5c22-61ec-4794-97b5-60fde553c36b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:40.019835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b5c22-61ec-4794-97b5-60fde553c36b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:40.021315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 21/42] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:40.027230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:40.027524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:40.027806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:40.047379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:40.048637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:40.166854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 22/43] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:40.167195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffcea6-e7e4-4ed7-9a79-93efb03c6bdb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:40.192145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:40.218889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:40.219542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:40.220116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:40.220116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:40.220862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:40.222268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:40.248545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 22/44] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Aug 4 10:11:40 2022] POST /identity/v3/roles => generated 223 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:11:40.343475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:40.347550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13598a-fdae-4ec3-9005-2bbc73dd0a03 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:40.387210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:40.388278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:40.388853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:40.389342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:40.390751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:40.392507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3b1f2-6ed2-424e-a1e0-5ab2b4734334 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:40.394407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 23/45] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:40.400418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:40.403555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:40.509829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4df80-75ad-4f59-85b6-94359c0465a9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:40.543275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:40.543897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:40.544211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:40.544491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:40.545458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:40.546890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e80c4-a1c4-4fe6-ae61-9dcde9042b43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:40.581695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 23/46] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:11:40 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:41.722945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef93d-6bc2-417a-84ed-83071762dd2e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:41.723457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef93d-6bc2-417a-84ed-83071762dd2e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:41.723539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef93d-6bc2-417a-84ed-83071762dd2e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:41.725405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 24/47] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:41.730607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:41.731038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:41.731169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:41.750952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:41.752175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:41.801633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:41.820568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 24/48] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:41.843275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ef93d-6bc2-417a-84ed-83071762dd2e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:41.946840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ef93d-6bc2-417a-84ed-83071762dd2e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:41.983372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:41.984558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:41.984850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:41.985131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:41.996186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:42.001429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:42.002392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:42.003262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 25/49] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:42.008556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.105604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07845c6c-a8ab-4e11-ae4a-dfa06c9caa3f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.138122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.138740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.139025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.139302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.140050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:42.140361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.141749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:42.150322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 25/50] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:42.155812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.184583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:42.260481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83f41b-e98e-4419-bfa7-a41af5808713 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.294473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.295763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.296289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.296786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.298816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.301609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:42.304766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 26/51] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:42 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:42.318091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.364826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:42.419100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4854-abaa-4e0b-8f9a-48bd30561fcc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.452149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.452778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.453061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.453331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.454468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.455909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a6f5b-14cd-4da6-ac8e-04478c837ce3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:42.457665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 26/52] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:42.464045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.552958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:42.558969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbb805-cdc2-4014-b72e-5d91eb2d0ceb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.581058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd9b6-7fc7-4228-b69f-7d04690b9e4b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.581351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd9b6-7fc7-4228-b69f-7d04690b9e4b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.581631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd9b6-7fc7-4228-b69f-7d04690b9e4b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.582978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 27/53] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:11:42.588488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.588779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.589050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.593933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.594549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.594829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.595150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.595944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.domain.id=default, user.id=bca06f724a4f42d0afd72d25da1e0911) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:42.596290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.597710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:42.604755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 27/54] 10.222.0.72 () {56 vars in 1322 bytes} [Thu Aug 4 10:11:42 2022] GET /identity/v3/role_assignments?user.id=bca06f724a4f42d0afd72d25da1e0911&scope.domain.id=default&role.id=abf8a86280c54a0798604e6c88e690cf => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:42.607146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:42.608416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:42.680707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 28/55] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:42.704445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.806015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c300b5-dfac-405a-a81e-a06bf856bbca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.838588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.839219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.839499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.839768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.840907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.842346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596feae-6980-4d3b-b601-36f19aea29ae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:42.843904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 28/56] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:42.849908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:42.893524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:42.945686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ab75e-7f38-4ba4-a5cb-3ba59e9a64bd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:42.978124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:42.978727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:42.979008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:42.979278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:42.979973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:42.981404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c51b9-801f-4403-af25-10d01110caa5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:43.007097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 29/57] 10.222.0.72 () {60 vars in 1140 bytes} [Thu Aug 4 10:11:42 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:44.988948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ca4d-017b-4468-93a7-57f1c9a81f88 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:44.989465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ca4d-017b-4468-93a7-57f1c9a81f88 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:44.989548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ca4d-017b-4468-93a7-57f1c9a81f88 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:44.991467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 29/58] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:44.996516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:44.996903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:44.997079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:45.014712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:45.015898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:45.080048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 30/59] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:45.104683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ca4d-017b-4468-93a7-57f1c9a81f88 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:45.131871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:45.202785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ca4d-017b-4468-93a7-57f1c9a81f88 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:45.235116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:45.235744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:45.236056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:45.236339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:45.237486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:45.238938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4befdc6-74ed-419e-baa5-38d9b57aa3cc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:45.240616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 30/60] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:45.246166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:45.272063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:45.341757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3b0c-8107-4e91-b953-9365cb234f26 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:45.373883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:45.374486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:45.374768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:45.375042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:45.375720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:45.377192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c7fc5-258c-4475-9ce7-d313c2234e6c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:45.403490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 31/61] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:11:45 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:48.589507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62d85-0355-457f-95d7-b547b223ac8b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.590352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62d85-0355-457f-95d7-b547b223ac8b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.590801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62d85-0355-457f-95d7-b547b223ac8b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.592345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 31/62] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:48.597651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.598422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.598844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.624991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:48.626432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:48.704524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761218ef-9b9d-4386-bd00-53c792cec97f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.705035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761218ef-9b9d-4386-bd00-53c792cec97f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.705462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761218ef-9b9d-4386-bd00-53c792cec97f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.706972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 32/63] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:48.712719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c243-7ac2-4d25-8486-2d8cc6edd8b9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.713285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c243-7ac2-4d25-8486-2d8cc6edd8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.713765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c243-7ac2-4d25-8486-2d8cc6edd8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.740957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 32/64] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:48.741829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2689c243-7ac2-4d25-8486-2d8cc6edd8b9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:48.743037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2689c243-7ac2-4d25-8486-2d8cc6edd8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:48.773025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:48.796291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:48.865453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 33/65] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:48.867291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61914-75fc-437e-b5c6-481a863827cd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.867800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61914-75fc-437e-b5c6-481a863827cd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.868295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61914-75fc-437e-b5c6-481a863827cd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.869759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 34/66] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:48.874978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.875481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.876018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.904498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:48.906418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:48.915407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731e0a0-9667-4ad5-9f5f-fc15367d97b1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:48.933447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:48.953542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:48.954576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:48.955058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:48.955531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:48.967463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:48.973483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:48.974343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:48.975731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:48.977599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 33/67] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:48.978613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:48.993773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 35/68] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:48.999109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.064282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.093785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946e146-f0a3-4721-a87f-d595871fe0b0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.104644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019a73a-a6df-44f1-8c98-293428817b6f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.127728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.128334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.128631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.128899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.137566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.138184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.138466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.138740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.139453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:49.139560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:49.139754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.141170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc5dde-99d2-4570-992b-344a52b83d3b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-999f2f91-69ab-4e33-9196-8a9b01ec0aa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:49.144575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:49.145398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:49.146123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 34/69] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:49.148083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feb18a-fe58-489c-9216-62c6d8b1a7ee None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.148385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feb18a-fe58-489c-9216-62c6d8b1a7ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.148674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feb18a-fe58-489c-9216-62c6d8b1a7ee None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.150129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 35/70] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:49.150690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feb18a-fe58-489c-9216-62c6d8b1a7ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 36/71] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab961a6e-3160-4ce7-9d64-78523bf6aa48 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab961a6e-3160-4ce7-9d64-78523bf6aa48 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab961a6e-3160-4ce7-9d64-78523bf6aa48 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 37/72] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:49.156808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab961a6e-3160-4ce7-9d64-78523bf6aa48 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.175760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.259074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab961a6e-3160-4ce7-9d64-78523bf6aa48 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.259198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feb18a-fe58-489c-9216-62c6d8b1a7ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.288701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.292902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.293523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.293804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.294075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.294788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:49.294874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.295093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.295676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.295986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.296258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.296755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4f89e-5094-4ad8-93e2-efc5dde05b03 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.305509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 38/73] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.306652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a191-5fde-4056-8851-869330d80c3c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.306826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:49.306932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a191-5fde-4056-8851-869330d80c3c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.307194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a191-5fde-4056-8851-869330d80c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:49.311750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:49.312592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:49.313482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 36/74] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:49.316809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.326901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1125a191-5fde-4056-8851-869330d80c3c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:49.328410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1125a191-5fde-4056-8851-869330d80c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:49.394588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 39/75] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:49.397815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.398105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.398378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.415172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:49.416374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:49.419164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.419468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636e35-53c4-43c4-b9c4-53de735d1c41 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.453560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.454200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.454483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.454768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.469591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.471050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.472719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 37/76] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.473506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.482823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 40/77] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:49.483674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.526287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.575478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaf3f8-9730-404e-8d65-5ee812a98689 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.583557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23a0f-2220-4f7a-9174-c3c3957909c2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.609121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.609761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.610053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.610309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.611461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.612929 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.614637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 38/78] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.615901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.622041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.622667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.622950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.623225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.623945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:49.624288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.625659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.634230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 41/79] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.635842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.638458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.720729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c476b-84c5-428d-912c-d62456793c46 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.732308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86019b03-fe7d-47d4-82be-9254a836344e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.737190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.754859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.755497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.756343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.756626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.757794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.759234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.760849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 39/80] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:49.761603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.765999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.766615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.766899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.767170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.833231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:49.834689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:49.836505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:49.862808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861366bb-88e9-41dc-a7f1-44d5a2871a86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:49.895920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:49.896611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:49.896888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:49.897159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:49.906794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:49.911724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:49.913247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:49.914208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 40/81] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:49.919261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.933852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 42/82] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:11:49.934679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:49.969942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.022467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedef063-e15d-4a85-adef-8009c7c4fc1d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.029754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1523fc2-d9f0-474a-9662-be7396f3bd91 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.031241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.057557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.058175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.058459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.058734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.064506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.065140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.065420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.065692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.066845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.068432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.070022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 43/83] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.070744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.124457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.125915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.166525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62678c8a-3a5e-4b45-ac02-b879694a0571 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.169383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.200873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.201479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.201760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.202035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.202715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.204159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.225421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 41/84] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:11:50.226237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.232395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 44/85] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:11:50 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:50.233137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.265419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.380459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f16e6-1d43-47d2-9ea4-99c77f9e553f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.390322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d9dd-bf34-4ec0-8195-2ebd5a4f5b93 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.429424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.430054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.430379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.432998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.432998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:50.432998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.433226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.439162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.439817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.440161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.440408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.441579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.442058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 42/86] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.442836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.443153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.444907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 45/87] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.445664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.463596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.568450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.576949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc429-76d9-4a73-81e5-66241e614bce None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.589519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c0a9-3daf-4a12-9579-0924767aa44a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.618269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.619328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.619799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.620174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.621544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.623187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.625292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 46/88] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.625949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.632853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.634049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.634481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.634961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.724718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.726468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.738452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.742348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2429-467d-43ef-b594-1af2b5120444 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.787563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.788499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.788965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.789856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.791553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.794004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.796630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 47/89] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.797809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.840291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:50.845954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 43/90] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:11:50.847070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.928992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc42b9-b7d3-430e-afb4-b668bef2653d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.963322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29d27-b9a9-4c73-a24b-e4fe45394475 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:50.964746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:50.965464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:50.965876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:50.966231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:50.967338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:50.967467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:50.969048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:50.975260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 48/91] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:50.975260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:50.984334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:51.042092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.042724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.043003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.043273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.044152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=bca06f724a4f42d0afd72d25da1e0911) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:51.044470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.047576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.061787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 44/92] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/role_assignments?user.id=bca06f724a4f42d0afd72d25da1e0911&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:51.062961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:51.128270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:51.139984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5a502-c4be-4fc4-8120-2b6d5530cd30 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:51.178644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.179494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.179722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.180027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.263772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.265953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.278923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38aabe-8bd3-495b-8bf7-a38d1f15eaed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:51.345307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.346463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.346964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.347490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.348977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.351330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.359984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 45/93] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:51 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:51.361411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:51.378205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:51.387325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 49/94] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:11:51.393614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:51.446556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:51.510748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d29e1-17df-4fd0-ae71-4ffc663f5d21 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:51.557939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.559055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.559344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.559616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] PATH_INFO: `/v3/projects/b1357390a6dd4ffda6907a49b528e972` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.560959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] RBAC: Authorizing `identity:get_project(project_id=b1357390a6dd4ffda6907a49b528e972)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.562333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ebd64-d14b-4b4e-b467-08c97b50e6b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:51.562446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.564437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 50/95] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:51 2022] GET /identity/v3/projects/b1357390a6dd4ffda6907a49b528e972 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:51.571009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:51.628078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.628758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.629078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.629417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.630216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:51.630558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.632146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9bd90-9ab2-4156-9ced-bf58208cd3b7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.642177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:51.657678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 46/96] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:11:51 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=9a068445d06b48629659d5184c3e91bc => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:51.714402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5eea7-e539-424a-b1a7-f44acefe74d8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:51.766618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:51.767254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:51.767535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:51.767809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:51.768665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=b1357390a6dd4ffda6907a49b528e972, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:51.768977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:51.770542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0816ec8-e9bd-436d-9633-7bce94eb1c48 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:51.777507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 51/97] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:11:51 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=b1357390a6dd4ffda6907a49b528e972&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:52.806014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9dcb0-133f-489d-9741-f92794d930a5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:52.807147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9dcb0-133f-489d-9741-f92794d930a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:52.807147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9dcb0-133f-489d-9741-f92794d930a5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:52.808153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 47/98] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:52.824664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e975d-749f-4f30-8848-ae5ba1309980 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:52.825148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e975d-749f-4f30-8848-ae5ba1309980 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:52.825246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e975d-749f-4f30-8848-ae5ba1309980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:52.852902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1e4e975d-749f-4f30-8848-ae5ba1309980 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:52.854112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1e4e975d-749f-4f30-8848-ae5ba1309980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:52.942211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 52/99] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:52.989534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9dcb0-133f-489d-9741-f92794d930a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.124766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9dcb0-133f-489d-9741-f92794d930a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:53.126216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68405f9-c76b-4400-9ddd-d94bd42fe4c7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.127161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68405f9-c76b-4400-9ddd-d94bd42fe4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.127878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:53.130028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68405f9-c76b-4400-9ddd-d94bd42fe4c7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.137638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 53/100] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:53.155233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.155928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.156459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.180807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:53.180807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.180807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.180807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.185950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:53.196430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:53.196826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:53.202280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:53.203619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:11:53.204895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:53.208338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 48/101] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:53.213136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.335548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:53.366590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 54/102] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:53.408840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.413912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bfe28-9ee0-41c4-87bc-829253fe5664 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:53.469215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:53.469850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.470140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.470425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.471210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:53.471527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:53.472926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:53.491893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 49/103] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:53.497389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.549436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:53.574425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c3e1f-c61f-4ffb-b904-4261d9f598f1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:53.627456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:53.628129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.628418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.628695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.632668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:53.634133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:53.639989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 55/104] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:53.641541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.708064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dd2a3-b81c-4052-8c1e-a0b6dab94aa8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:53.716416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:53.784989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217817f5-731e-45ea-9b44-c3e10dc75a4c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:53.785972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:53.787419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.787902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.788409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] PATH_INFO: `/v3/users/d91d9e95ea8a4530b676229d9bb0ff90` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.817160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] RBAC: Authorizing `identity:get_user(user_id=d91d9e95ea8a4530b676229d9bb0ff90)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:53.819450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:53.821533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 50/105] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:53 2022] GET /identity/v3/users/d91d9e95ea8a4530b676229d9bb0ff90 => generated 289 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:53.831477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:53.861399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:53.862060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:53.862315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:53.862721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:53.863444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:53.865643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:53.926280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 56/106] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:11:53 2022] POST /identity/v3/projects => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:11:53.986499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:54.009353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7141a00-ad47-4d99-8eb0-6c85e6cd2202 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:54.066383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:54.067390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.067942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.068476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.143163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:54.145116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b3063-918d-49b5-91a5-c0a6b005786d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:54.252511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:54.275734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 51/107] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:11:54.279272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:54.422061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64622952-61d9-4e7f-8e10-23f73c503013 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:54.424098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b253b1-fa9d-4268-8afb-739ca4759190 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.424401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b253b1-fa9d-4268-8afb-739ca4759190 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.424801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b253b1-fa9d-4268-8afb-739ca4759190 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.426346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 52/108] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:54.430612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:54.432536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492674-ad97-4be3-8834-c16ee4cd6e45 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.432962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492674-ad97-4be3-8834-c16ee4cd6e45 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.433510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492674-ad97-4be3-8834-c16ee4cd6e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.462902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:54.463972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.465293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.465783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.467267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:54.469011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:54.469122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ac492674-ad97-4be3-8834-c16ee4cd6e45 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:54.470332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac492674-ad97-4be3-8834-c16ee4cd6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:54.476278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 57/109] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:54 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:54.478922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:54.563179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 53/110] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:54.564512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0449557-ac24-49fa-abe6-8b16f4c51ad6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.564806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0449557-ac24-49fa-abe6-8b16f4c51ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.565081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0449557-ac24-49fa-abe6-8b16f4c51ad6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.566350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 54/111] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:54.572921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.572921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.572921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.593009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:54.594200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:54.612517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020024d-0ea1-4b2c-87a8-ac4e367a602c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:54.660079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:54.661540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.662491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.662907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.664191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=d91d9e95ea8a4530b676229d9bb0ff90) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:54.664669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:54.666955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:54.688479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 58/112] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:11:54 2022] GET /identity/v3/role_assignments?user.id=d91d9e95ea8a4530b676229d9bb0ff90&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:54.689243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:54.707914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 55/113] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:54.758399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:54.762022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:54.840633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:54.861583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea1ad-632c-433f-8d94-5825c500571a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:54.898115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:54.898789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.899075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.899353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.911448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6561ae2a-751d-4950-ae2b-2ab18529069c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:54.920661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:54.921621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:54.922355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:54.925853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 59/114] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:54.927729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8be9-f676-4680-b70a-c77debb8a27a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.928676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8be9-f676-4680-b70a-c77debb8a27a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.929105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8be9-f676-4680-b70a-c77debb8a27a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.931141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 60/115] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:54.933154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e423-75a3-4e78-9663-ea11c26a1d68 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.933609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e423-75a3-4e78-9663-ea11c26a1d68 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.934029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e423-75a3-4e78-9663-ea11c26a1d68 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:54.935815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 61/116] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:54.936717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e423-75a3-4e78-9663-ea11c26a1d68 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:54.936982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb064061-51d9-4e91-8485-9e48dc63b9cd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:54.992056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:54.993223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:54.993692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:54.994146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.007660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ad40c5bd-febd-4503-8bf7-bf951befdce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:55.014067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:55.014920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:55.015690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:55.017702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 56/117] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:55.019272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cccc0-274a-475f-88a7-2f77253fed69 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.019717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cccc0-274a-475f-88a7-2f77253fed69 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.020162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cccc0-274a-475f-88a7-2f77253fed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.045492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0e1cccc0-274a-475f-88a7-2f77253fed69 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:55.047359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0e1cccc0-274a-475f-88a7-2f77253fed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:55.090190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.095868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e423-75a3-4e78-9663-ea11c26a1d68 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.133864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 57/118] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:55.135400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.135663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.135947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.140950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.141836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.142578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.143979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.144918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:55.145442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.147694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.155558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:55.156942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:55.158423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 62/119] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.160622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.191151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.253325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 58/120] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:55.254468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.302040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5f832-6c5b-4160-9d7e-fc6d6c315443 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.340116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.340452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.340675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.340957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.341678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:55.342047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.343512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.352874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 63/121] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.360017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.387114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8c4a0-a67b-49fd-bbdc-85bdfa51afc9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.416383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.429849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.430651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.431032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.431432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.432756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.434386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.436204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 59/122] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.437065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.471366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.502521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b9991-f797-4fb3-8b83-1862274aa14e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.550305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76458b-43b1-43bb-8e64-d43be5bb7400 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.551396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.553694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.555679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.556170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.570915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:55.576923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:55.577781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:55.578837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 64/123] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:55.580511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.597714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.598367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.598650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.599328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.614407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:55.621594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:55.623431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 60/124] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:55.624902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.676992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.701074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41619499-3ba2-44a1-8b05-fbf2ef14312b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.740854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.748897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.749999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.750428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.750893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.751414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f825940-1c94-4413-91dc-684e59c81717 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.752383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.755135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.757024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 65/125] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.757914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.801532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.802172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.802485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.802758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.804488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.806428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.808835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 61/126] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.812261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.888759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1994077-c5f8-441b-bf92-f3c6ff83f1b0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.931203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.931836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.932142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.932423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.933153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:55.933458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:55.933835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6bd8-6e48-4150-ac2d-fa916cf56972 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:55.934880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:55.945265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 66/127] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:55.946153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:55.964071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:55.994638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:55.995648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:55.996636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:55.997137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:55.998483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:55.999254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.001141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.014007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 62/128] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.015808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.050247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.061493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bf316-a9d2-492b-b66d-01011b7698bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.099993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.100645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.100928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.101202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.102364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.103990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.111003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 67/129] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:55 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.111441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.157252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99af4-a36b-43dd-9503-6c241de41681 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.197516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.204635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.205580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.206063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.206966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/bca06f724a4f42d0afd72d25da1e0911/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.210444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, user_id=bca06f724a4f42d0afd72d25da1e0911, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.212676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.219872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959352-e0e0-4350-a92a-d05df7b2d878 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.230210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 63/130] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:11:56 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/bca06f724a4f42d0afd72d25da1e0911/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:11:56.231455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.304168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.304887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.305176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.305454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.308614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.314685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.316680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 68/131] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:56 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.321797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.415861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.442471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc0a1c-c52b-48d8-8927-f4300de18dc4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.481722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.483466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.484371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.485262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.487936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.489565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104f157-3078-413a-b5cc-69d47c467ae6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.492089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.495629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 64/132] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:56 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.497433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.526223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.526952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.527242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.527521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/a701fdc63c4941aab5144f692c2e4010/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.529795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, user_id=a701fdc63c4941aab5144f692c2e4010, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.531297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.550887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 69/133] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:11:56 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/a701fdc63c4941aab5144f692c2e4010/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:11:56.556976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.578215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.623369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793546b-a15c-4d81-ae14-92bdb2cb15d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.662426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.663447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.663971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.664451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] PATH_INFO: `/v3/projects/b1357390a6dd4ffda6907a49b528e972` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.665958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] RBAC: Authorizing `identity:get_project(project_id=b1357390a6dd4ffda6907a49b528e972)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.667678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03c41-ae18-446c-8d83-741b3150ed10 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.671664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 65/134] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:56 2022] GET /identity/v3/projects/b1357390a6dd4ffda6907a49b528e972 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.673276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4913a5-029d-4705-a383-c7a3562e0c0c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.673768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4913a5-029d-4705-a383-c7a3562e0c0c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.674253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4913a5-029d-4705-a383-c7a3562e0c0c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.675674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 66/135] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:56.683981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.683981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.683981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.710581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:56.712311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:56.728748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.745050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866857a-ac69-49a0-a013-d49aae1c6246 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.785942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.787121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.787430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.787710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:56.789046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:56.790506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:56.792588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 70/136] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:56 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:56.793462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.807666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 67/137] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:56.808782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:56.905963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:56.930629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e08d-d7c8-4418-ad9f-5dbdcae0d5e3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.957068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2170780-9b6b-4d01-8897-b77854c0def4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:56.992287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:56.992984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:56.993269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:56.993548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/a701fdc63c4941aab5144f692c2e4010/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.000408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, user_id=a701fdc63c4941aab5144f692c2e4010, role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.001886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.018315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 68/138] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:11:56 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/a701fdc63c4941aab5144f692c2e4010/roles/9a068445d06b48629659d5184c3e91bc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:11:57.019296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:57.020586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.022105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.022762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.023391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] PATH_INFO: `/v3/projects/b1357390a6dd4ffda6907a49b528e972/users/a701fdc63c4941aab5144f692c2e4010/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.026261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1357390a6dd4ffda6907a49b528e972, user_id=a701fdc63c4941aab5144f692c2e4010, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.028329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559f5d0-7255-483f-91df-95d27e8fbb46 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.032278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:57.066958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 71/139] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:11:56 2022] PUT /identity/v3/projects/b1357390a6dd4ffda6907a49b528e972/users/a701fdc63c4941aab5144f692c2e4010/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:11:57.068706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a23cb1-eadb-4e0f-9f8c-6d30940c8620 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.069000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a23cb1-eadb-4e0f-9f8c-6d30940c8620 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.069281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a23cb1-eadb-4e0f-9f8c-6d30940c8620 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.073350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 72/140] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:57.079127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.079415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.079742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.125517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:57.126785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:57.229892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad21dab-1183-4abc-843f-c71912263da3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:57.276837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 73/141] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:11:57.287658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.288346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.288637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.288921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.290118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.291591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.296526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 69/142] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:11:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:57.299752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:57.320289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:57.392842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:57.435321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb13850-07ba-4559-bf15-d1e0f71f2c45 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:57.452048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:57.488649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.489722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.490260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.490741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.491661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.493416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.499877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60efe9-1ff5-4bf0-a056-2e12abc412bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:57.529948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 74/143] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:11:57 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:11:57.560977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.562013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.562521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.562995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.583195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:57.589337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:57.590254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:11:57.591041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:57.597146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 70/144] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:11:57.603503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:57.736002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761b83e-3c90-4d86-b50b-190622f4492c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:57.761999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:57.775799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.776662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.776950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.777221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.778216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:57.778539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.779970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.789417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 75/145] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:57.795162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:57.917286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1338f-c073-429f-8882-e1a7b21a877f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:57.973281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:57.974012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:57.974335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:57.974616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] PATH_INFO: `/v3/users/d91d9e95ea8a4530b676229d9bb0ff90` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:57.975924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] RBAC: Authorizing `identity:get_user(user_id=d91d9e95ea8a4530b676229d9bb0ff90)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:57.977462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:57.979375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 71/146] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:57 2022] GET /identity/v3/users/d91d9e95ea8a4530b676229d9bb0ff90 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:57.984999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:58.110854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b726-ae5b-434c-8c75-024e33422575 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:58.151651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:58.152349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:58.152629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:58.152904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:58.154069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:58.155520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ad43d-a403-4210-a139-a27839c7c931 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:58.157516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 76/147] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:11:57 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:58.163886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:58.167880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:58.278375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bdd1a-6f85-4192-af47-f95a053452f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:58.322985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:58.324052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:58.324545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:58.325021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/d91d9e95ea8a4530b676229d9bb0ff90/roles/d28acb0ee63a4d7d87658a048960d8fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:58.327216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, user_id=d91d9e95ea8a4530b676229d9bb0ff90, role_id=d28acb0ee63a4d7d87658a048960d8fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:58.328884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb901b6-58c1-4201-8511-18a8934f4d8a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:58.344902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 72/148] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:11:58 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/users/d91d9e95ea8a4530b676229d9bb0ff90/roles/d28acb0ee63a4d7d87658a048960d8fb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:11:59.008249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323900-3bdf-4124-a5d1-f14078d1ff86 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.008778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323900-3bdf-4124-a5d1-f14078d1ff86 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.008778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323900-3bdf-4124-a5d1-f14078d1ff86 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.010326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 77/149] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:59.016384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58402be-75ba-4ac1-a3f2-cc167a203d59 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.016671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58402be-75ba-4ac1-a3f2-cc167a203d59 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.016950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58402be-75ba-4ac1-a3f2-cc167a203d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.040978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e58402be-75ba-4ac1-a3f2-cc167a203d59 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:59.042175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e58402be-75ba-4ac1-a3f2-cc167a203d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:59.103871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.112109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 73/150] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:59.153141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323900-3bdf-4124-a5d1-f14078d1ff86 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.195986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc26e6-42d4-49e0-952b-f2bd97531bfa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.196937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc26e6-42d4-49e0-952b-f2bd97531bfa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.197411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc26e6-42d4-49e0-952b-f2bd97531bfa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.198981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 74/151] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:11:59.204366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.204919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.205402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.219335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.228032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:11:59.229658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:11:59.281501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323900-3bdf-4124-a5d1-f14078d1ff86 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.311714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 75/152] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:11:59.329244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.329905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.330186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.330466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.336998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.353910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:59.359381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:59.360294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:11:59.361043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 78/153] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:11:59.366644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.450199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b488-f0ab-41ed-814c-dcd623ccb1c6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.477884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.479180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e0088-dcbd-4d18-a414-b5cffed62e4c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.491328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.492406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.492732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.493011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.494559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.506796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:11:59.512647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:11:59.513692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:11:59.514649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:11:59.516982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 76/154] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:11:59.517878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.518506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.518785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.519058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.519781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:59.522402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:59.522402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:59.523747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.531196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 79/155] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:59.537455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.638404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366368a-bfe4-4147-887a-ccbd56e18084 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.673372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3d3a9-8673-430c-bd2a-07616c79911a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.676338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.677255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.677745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.678214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.679137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:11:59.679661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:59.681387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:59.691188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 77/156] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:59.697328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.709719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.710368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.710649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.711025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.712251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:59.713907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:59.715990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 80/157] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:11:59.721442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:11:59.765585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.769763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:11:59.830801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3d29-0bf2-458f-8059-61e4a7fda846 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.833467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2e485-9dac-4f8c-b355-eba038d5a5c8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:11:59.890331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.891099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.891454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.891798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] PATH_INFO: `/v3/users/bca06f724a4f42d0afd72d25da1e0911` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.893094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] RBAC: Authorizing `identity:get_user(user_id=bca06f724a4f42d0afd72d25da1e0911)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:11:59.894710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:11:59.900371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 78/158] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/users/bca06f724a4f42d0afd72d25da1e0911 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:11:59.901240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:11:59.902056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:11:59.902587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:11:59.903022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:11:59.905832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.014007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:00.015879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:00.065969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:00.118261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62c2d2-9f0b-49b2-8d2d-cbd1d760b54b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:00.135894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 81/159] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:00.145019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.149027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:00.197646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:00.199077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.199565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.200065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.265878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb2f49-fbb3-46ac-8d6b-9a58ec715f62 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:00.319211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:00.319870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.320176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.320453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.325115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:00.326823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:00.329026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 82/160] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:00 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:00.343647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.358469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:00.392895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:00.394747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71def58d-af46-4c12-b394-e4b4ba3557e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:00.475015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62aa6f-e7e4-4ce5-b32d-5a81f20f57c5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:00.519796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:00.520455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.520739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.521579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.527976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:00.527976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:00.527976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6624d7d-8e3c-44be-aa4d-84c22aca6f2f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:00.531394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 83/161] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:00 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:00.607589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448797-de31-4e87-92bd-0c877e5d5d2b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.607920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448797-de31-4e87-92bd-0c877e5d5d2b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.608626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448797-de31-4e87-92bd-0c877e5d5d2b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.610427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 84/162] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:00.612713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 79/163] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:00.618747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448797-de31-4e87-92bd-0c877e5d5d2b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.627173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.627806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.628317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.660763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:00.663447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:00.680531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:00.738700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:00.787348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448797-de31-4e87-92bd-0c877e5d5d2b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:00.792602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 80/164] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:00.835942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:00.836630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:00.836914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:00.837191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:00.840690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:00.842238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:00.843970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 85/165] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:00 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:00.850719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.853489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:00.986811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefa4ca-3e2f-4bce-95f0-3896c9e3c911 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:01.014482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f762a2-72b1-4279-9897-cbe6ea6a909b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:01.041202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:01.041870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.042156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.042433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.073541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:01.073858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:01.074448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.074725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.074993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.075788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=bca06f724a4f42d0afd72d25da1e0911) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:01.076120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:01.077564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:01.079074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:01.079935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:01.080949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:01.084392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 86/166] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:01.090014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:01.092797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 81/167] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:00 2022] GET /identity/v3/role_assignments?user.id=bca06f724a4f42d0afd72d25da1e0911&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:01.100391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:01.225344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e9aa-115c-4c94-807f-fedd80bc9032 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:01.275142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:01.275781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.276088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.276366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.277089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:01.277393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:01.278787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:01.287883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 87/168] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:01.297089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:01.312889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:01.436576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422b3-104f-4839-95c4-d02bdf0c3648 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:01.487205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:01.487860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.488170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.488441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.490405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:01.491897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d83c6a-a84a-4385-82aa-2d139b259263 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:01.493629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 82/169] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:01 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:01.499418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:01.600922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:01.648050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c27fe-c73b-468d-b5e4-b81eb256ab47 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:01.651504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062bbb-d588-47ed-8522-c334feb6642c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.651786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062bbb-d588-47ed-8522-c334feb6642c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.652083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062bbb-d588-47ed-8522-c334feb6642c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.653582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 83/170] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:01.660998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.661283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.661583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.694007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:01.698497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:01.733893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:01.734518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:01.734797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:01.735066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:01.821239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 84/171] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:01.839556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:01.841077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:01.858866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:01.870400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:01.965037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:02.042738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 88/172] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:02.049380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.054139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965d1c5-960c-4d6d-987f-5e289b714528 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:02.114072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:02.115240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.115529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.115802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.133130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:02.138071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:02.138936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:02.139696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 85/173] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:02.146660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.268310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:02.278136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1816891-51bf-47bc-af54-225733d44b2c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:02.320011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd50b7-20f6-4877-973c-cf3143a698ff None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:02.367059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:02.368195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.368688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.369155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.372582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:02.374389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:02.376554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 89/174] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:02.382122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:02.382745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.383021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.383289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.384029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:02.384332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:02.385685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:02.399467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.400181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 86/175] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:02.416269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.456322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:02.537988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a7d8-0951-4b32-abe9-9cac5a3b0b83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:02.574068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:02.574692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.574971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.575243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.576132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:02.576441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:02.577949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49d0cf-0456-4fb9-b980-b811d520d630 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:02.585136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 90/176] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=9a068445d06b48629659d5184c3e91bc => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:02.586575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f703-d59e-491a-9232-a253c4fff203 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.586864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f703-d59e-491a-9232-a253c4fff203 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.587144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f703-d59e-491a-9232-a253c4fff203 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.592593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 91/177] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:02.594284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe23174-f513-440b-aac9-5708d4aea096 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.594567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe23174-f513-440b-aac9-5708d4aea096 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.594840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe23174-f513-440b-aac9-5708d4aea096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.617762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bbe23174-f513-440b-aac9-5708d4aea096 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:02.618871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bbe23174-f513-440b-aac9-5708d4aea096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:02.682287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674a9c0-7b08-434a-b4fd-8a90e6d6d6c9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:02.696804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 92/178] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:02.700722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9311-3daf-4c89-869b-e78b032c68fa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.701257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9311-3daf-4c89-869b-e78b032c68fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.701731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9311-3daf-4c89-869b-e78b032c68fa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.703537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 93/179] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:02.704844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39e11-ce94-45b9-ac33-3a15ac8713ea None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.705334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39e11-ce94-45b9-ac33-3a15ac8713ea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.705807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39e11-ce94-45b9-ac33-3a15ac8713ea None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.707201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 94/180] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:02.713037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.713620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.714117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.739048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:02.739810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:02.740496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:02.752064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:02.752766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.753086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.753364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] PATH_INFO: `/v3/users/a701fdc63c4941aab5144f692c2e4010` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.756385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] RBAC: Authorizing `identity:get_user(user_id=a701fdc63c4941aab5144f692c2e4010)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:02.757853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b266dd-5eb2-4297-b64a-ce615d507f03 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:02.760217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 87/181] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/users/a701fdc63c4941aab5144f692c2e4010 => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:02.762034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:02.762444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:02.762719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:02.781223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:02.796800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:02.798558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:02.835879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 95/182] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:02.839228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.887783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 88/183] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:02.892062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:02.969237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76841e-f2d0-4db6-b849-a6735bfee1e5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.005294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.006427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.006706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.006976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.007465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d759ecb-c02c-41d4-b477-f33222695082 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.018537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:03.023551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:03.024459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:03.025243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 96/184] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:03.030176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.037837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.047724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.048385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.048666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.048939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.050935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.123501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.125005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.150066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97ae0e-c17a-405f-a4d3-6a05a0033b9f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.183769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.184426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.184699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.184969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.195414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:03.200349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:03.201412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:03.202219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 97/185] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:03.202850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.224914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 89/186] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:03.225738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.253752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.288549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.311897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536aff9-bf36-486a-bb2c-2fefe7774373 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.326030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f583-4a4d-4c89-8fb7-94c34e6555b8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.345734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.346380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.346665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.346938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.357189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:03.358541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.359144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.359422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.359690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.360431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:03.360733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.362091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-615c2c28-86a4-4e17-9b95-d4a25f141f52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:03.362183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:03.363139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:03.364295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 98/187] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:03.365781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09037aa-91ac-41cb-a4c7-667c637ccaa6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.365781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09037aa-91ac-41cb-a4c7-667c637ccaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.365781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09037aa-91ac-41cb-a4c7-667c637ccaa6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.366935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 99/188] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:03.367468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09037aa-91ac-41cb-a4c7-667c637ccaa6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.379925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 90/189] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.379925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.462647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.472788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09037aa-91ac-41cb-a4c7-667c637ccaa6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.474851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0edf-4b02-49d4-8401-a921a9aa766c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.506714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.507342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.507580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.507719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.507855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.508166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.508453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.508601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:03.508727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] PATH_INFO: `/v3/projects/b1357390a6dd4ffda6907a49b528e972` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.508902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.509895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] RBAC: Authorizing `identity:get_project(project_id=b1357390a6dd4ffda6907a49b528e972)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.510289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acd01a-ebb6-4f94-af15-7614acbd4480 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.511317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.512568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.513013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 91/190] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/projects/b1357390a6dd4ffda6907a49b528e972 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.513763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.523422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 100/191] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.524742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.525037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.525309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.543634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:03.544846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:03.609679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 101/192] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:03.610491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.615808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7353e-4989-41a1-946f-91c0cc940d5b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.651181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.651799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.652114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.652391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.653107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:03.653414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.654779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.663157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 92/193] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.664027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.682922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.711871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40dbc-6adf-44a6-a0fe-5f34acfe416d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.737830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.746444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.747083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.747363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.747633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] PATH_INFO: `/v3/users/d91d9e95ea8a4530b676229d9bb0ff90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.749061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] RBAC: Authorizing `identity:get_user(user_id=d91d9e95ea8a4530b676229d9bb0ff90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.750503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.752127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 102/194] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/users/d91d9e95ea8a4530b676229d9bb0ff90 => generated 289 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.752870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.763042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf5ea-09a4-4574-bf48-13b9728defb3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.796688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.797289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.797569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.797839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.798615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=b1357390a6dd4ffda6907a49b528e972, user.id=a701fdc63c4941aab5144f692c2e4010) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:03.798922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.800369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.806074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 93/195] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/role_assignments?user.id=a701fdc63c4941aab5144f692c2e4010&scope.project.id=b1357390a6dd4ffda6907a49b528e972&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.806837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.861161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325a29-8c19-472a-953d-ff1c248a2246 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.899621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.900395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.900740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.901010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.901611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:03.917071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.918649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.920765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 103/196] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.921551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.927771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1779b3-4941-469e-81db-053ea4b6854b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:03.965904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:03.965904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:03.965904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:03.966218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:03.969385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:03.970158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:03.971756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 94/197] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:03.972582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:03.986820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.039020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4705dad-993b-4558-bcb6-134401b38017 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.072756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.073506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.073850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.074127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.075265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.076733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.078307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 104/198] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:04.079040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.084403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf7c86-f7a9-4d49-ad1f-dd8f754a73f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.117699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.118317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.118602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.118874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.137214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.178160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd0ad7-fae9-4513-8fde-e607d5994578 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.185323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.186799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.200950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.213279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.213898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.214178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.214458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.280729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.282212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.285149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 95/199] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:04.285961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.336283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.384021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 105/200] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:04.384021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.388881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0568a5d-d658-4344-802a-e0bec29072fa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.404030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.423782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.424431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.424704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.424972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.425649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.427081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.448893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 96/201] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:04 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:04.456986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.486653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba92b4-ec08-4460-959b-a89e4e0b9b3c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.526523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.527152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.527426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.527754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.583076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be619-7080-4e6d-a70e-91d8f4c0cb83 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.596077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.616165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.617871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.634717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.635381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.636332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.636332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.637155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.638944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.641190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 97/202] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:04.641190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.653774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.767726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 106/203] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:04.769000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.773632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380875ef-1d1d-40a3-ae7b-c9a0a33b15bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.815892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.816895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.817360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.817764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.819514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.821070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.823004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 98/204] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:04.823645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:04.872529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.897472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0130-d75a-48cb-b550-77d39e31ed39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.923034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:04.944042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e69d-8e88-49bd-8814-b21d2941b9b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:04.944744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:04.945379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:04.945661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:04.945968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:04.947097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c, user.id=d91d9e95ea8a4530b676229d9bb0ff90) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:04.947407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:04.949823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:04.957886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 107/205] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/role_assignments?user.id=d91d9e95ea8a4530b676229d9bb0ff90&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:04.960217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:05.014012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:05.015060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:05.015541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:05.016084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:05.018745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:05.020501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:05.023986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 99/206] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:05.036021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:05.148841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa29-b853-43a3-b803-10a535617f9c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:05.152439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9194d-d7bb-4ee7-b751-3924fa1e8d86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:05.180653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:05.194263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:05.194911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:05.195478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:05.196081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:05.196958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=18990279906c40b58ee724e89dcb45ca) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:05.197324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:05.198832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be8ff7-9e6c-4f23-9850-560e42ff031d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:05.205091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 100/207] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:05 2022] GET /identity/v3/role_assignments?user.id=18990279906c40b58ee724e89dcb45ca&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:05.222504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:05.223161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:05.223443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:05.223716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:05.224634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=18990279906c40b58ee724e89dcb45ca) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:05.225044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:05.226556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94387-d476-4aa2-b4c7-580d6644ea47 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:05.233508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 108/208] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:04 2022] GET /identity/v3/role_assignments?user.id=18990279906c40b58ee724e89dcb45ca&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=9a068445d06b48629659d5184c3e91bc => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:06.736275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48159463-e401-41e9-8451-f021a888e309 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:06.736790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48159463-e401-41e9-8451-f021a888e309 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:06.736896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48159463-e401-41e9-8451-f021a888e309 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:06.738334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 101/209] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:06.744502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:06.745396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:06.745874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:06.754620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc063-713e-4df2-8c92-008bedd1b42c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:06.754909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc063-713e-4df2-8c92-008bedd1b42c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:06.755860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc063-713e-4df2-8c92-008bedd1b42c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:06.757502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 102/210] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:06.762412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:06.762699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:06.762977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:06.767742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:06.768999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:06.781795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:06.783395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:06.830168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:06.854844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 109/211] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:06.855975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 103/212] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:06.878505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:06.879934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.003035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bcfa9-7fde-495e-857d-8577f1e79630 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.016657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f860-aa35-4bb0-b04a-6c0ea120a5fd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.019918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.041074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.041760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.042040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.042316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.054490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:07.056165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.057040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.057556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.058031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.059413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:07.059590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:07.060475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:07.061233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 110/213] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:07.064287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.066104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 104/214] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:07.071503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.074842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.117005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.187013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1278b-e057-4b6e-8177-1445d0a9d01d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.199895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1faa7-df94-44eb-9890-586aec4be3eb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.230347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.232052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.232544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.233285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.234718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.236561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfe23a-5f2a-4c57-b58b-d64ee62efb8a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.251152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.251779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.252291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.252408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.253175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:07.253416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.255772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.272167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 105/215] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:07 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:07.272492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 111/216] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:07.273440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.279573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee07495-08bb-417c-8557-4a31d0b883db None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.279936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee07495-08bb-417c-8557-4a31d0b883db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.280161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee07495-08bb-417c-8557-4a31d0b883db None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.281855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 106/217] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:07.289178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad962-1b53-44d6-8d22-e235a8aab0f8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.289597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad962-1b53-44d6-8d22-e235a8aab0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.289928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad962-1b53-44d6-8d22-e235a8aab0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.310536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ecad962-1b53-44d6-8d22-e235a8aab0f8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:07.311769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ecad962-1b53-44d6-8d22-e235a8aab0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:07.341535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.387681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fd941-8b85-4692-9f98-ff6c4eb93d6b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.389577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.404428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 107/218] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:07.404428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047b4e8-4ecb-45a7-b1cb-f3191551df51 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.404428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047b4e8-4ecb-45a7-b1cb-f3191551df51 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.404428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047b4e8-4ecb-45a7-b1cb-f3191551df51 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.404428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 108/219] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:07.413683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.414019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.414297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.435444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.436144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.436429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.436710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] PATH_INFO: `/v3/users/55f0ea4a51504563a788c1c44a2f02d5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.437503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:07.438690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:07.453335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] RBAC: Authorizing `identity:get_user(user_id=55f0ea4a51504563a788c1c44a2f02d5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.454825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.456628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 112/220] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/users/55f0ea4a51504563a788c1c44a2f02d5 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:07.457344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.518017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 109/221] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:07.519275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.575037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3bbb-c6d4-41f2-9187-d15e673de93a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.614702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.615351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.615652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.615926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.620664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.629290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:07.635602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:07.636504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 113/222] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:07.637279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.637971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415080-e979-428d-92c3-a65d9d43fa23 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.661234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.684839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.685474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.686024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.749203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802346b1-cf0c-482a-84be-eed62dcde2c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.774052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.775613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.793645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.794304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.794584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.794857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.811169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:07.818519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:07.819419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:07.820591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:07.821738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 114/223] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:07.822707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.897487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.914990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 110/224] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:07.916293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:07.918631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:07.941883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c848b7-db61-4639-91df-052b95380d53 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:07.983592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:07.984551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:07.984839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:07.985116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:07.985841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:07.986144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:07.987526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:07.999169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 115/225] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.000008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.041430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061b77-bc75-40e9-a82d-1bf30adbecb5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.077700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.078675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.079226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.079740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.080698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:08.081203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.082777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.092225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 111/226] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.093451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.128750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec9b55-653b-4db0-99ba-61a0390e6c3d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.158312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:08.172167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.172821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.173101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.173373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.175133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.176791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.179587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 116/227] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:07 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.180189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.195859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:08.220970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54684315-ee51-467f-a2c1-decbf6752c9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.261473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.263016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.263530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.264035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.265587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.268019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.270858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 112/228] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:08 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.271977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.307080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ba1f8-9c64-4ba6-bfe0-446d8581c87d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.347442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d703-8639-48b2-9550-d0515215998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.348151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d703-8639-48b2-9550-d0515215998a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.348433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d703-8639-48b2-9550-d0515215998a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.348767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d703-8639-48b2-9550-d0515215998a None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.350943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d703-8639-48b2-9550-d0515215998a None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.352739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d703-8639-48b2-9550-d0515215998a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.356032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 117/229] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:08 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.357016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d703-8639-48b2-9550-d0515215998a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.401494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230199ad-bb05-467b-86d0-4da5cbd71c39 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.423553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:08.448428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794698b-1aca-4529-8541-0913f297699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.449383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794698b-1aca-4529-8541-0913f297699b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.449869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794698b-1aca-4529-8541-0913f297699b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.450438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794698b-1aca-4529-8541-0913f297699b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.451421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794698b-1aca-4529-8541-0913f297699b None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=55f0ea4a51504563a788c1c44a2f02d5) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:08.451931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794698b-1aca-4529-8541-0913f297699b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.454168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794698b-1aca-4529-8541-0913f297699b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.455949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:08.462401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 113/230] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:08 2022] GET /identity/v3/role_assignments?user.id=55f0ea4a51504563a788c1c44a2f02d5&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.463846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794698b-1aca-4529-8541-0913f297699b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.486552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d703-8639-48b2-9550-d0515215998a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.573494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.574083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.574369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.574645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.579500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.581021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.582818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 118/231] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:08 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.588715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.593980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794698b-1aca-4529-8541-0913f297699b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.642093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.643148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.643672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.644300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.646422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.648355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.650729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 114/232] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:08 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:08.656962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:08.711069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:08.762289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae092565-e05e-4c17-89ea-82b663113c66 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.794646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289387f0-4050-4a16-ba50-a0a164e964a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:08.798062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.799250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.799727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.800272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/18990279906c40b58ee724e89dcb45ca/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.803185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user_id=18990279906c40b58ee724e89dcb45ca, role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.804850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16047-84cd-4da4-99eb-c02d7c54cc58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.820754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 115/233] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:08 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/18990279906c40b58ee724e89dcb45ca/roles/9a068445d06b48629659d5184c3e91bc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:12:08.853654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:08.854669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:08.855166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:08.855635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/18990279906c40b58ee724e89dcb45ca/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:08.857887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user_id=18990279906c40b58ee724e89dcb45ca, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:08.859678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c06947-0afd-4600-9b68-f97eabf0da3f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:08.879400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 119/234] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:08 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/18990279906c40b58ee724e89dcb45ca/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:09.650191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c2e0-65a4-4a98-acae-581f7daeb7f5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:09.650479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c2e0-65a4-4a98-acae-581f7daeb7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:09.650765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c2e0-65a4-4a98-acae-581f7daeb7f5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:09.652144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 116/235] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:09.657379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:09.657670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:09.657948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:09.673489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:09.685447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:09.686633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:09.760826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f166-0b5c-49bb-ad52-1c4ad5aa0575 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:09.761167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f166-0b5c-49bb-ad52-1c4ad5aa0575 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:09.761388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f166-0b5c-49bb-ad52-1c4ad5aa0575 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:09.763036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 117/236] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:09.769017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:09.769305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:09.769584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:09.773546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 120/237] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:09.797868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:09.804958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:09.806164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:09.913877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ecaaa-9221-460d-83c6-6f60f11fa418 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:09.951669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:09.952809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:09.953419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:09.953891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:09.956810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:09.968158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:09.974527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:09.976196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:09.976973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:09.979743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 121/238] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:09.985769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.022080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 118/239] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:10.092621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.120483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc34a3-3c28-4153-8d18-2c73d692cd1d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.157230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.158279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.158762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.159262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.160195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:10.160742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.162353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.171682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 122/240] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:10.178384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.205947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:10.242018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:10.289038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a66ea-aee0-4d66-ba9b-4df4afd9aae2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.325302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.326296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.326806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.327278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] PATH_INFO: `/v3/users/4949ac25ad1a4e598c4e31b21bd6c141` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.341439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] RBAC: Authorizing `identity:get_user(user_id=4949ac25ad1a4e598c4e31b21bd6c141)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.343215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.345253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 123/241] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:10 2022] GET /identity/v3/users/4949ac25ad1a4e598c4e31b21bd6c141 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:10.350903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.393918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caff5d-6d61-426b-9441-b36b35faeec2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.457083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3f6d-9abe-4de0-8114-b0694adec357 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.476302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:10.492678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.493356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.493640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.493915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.496308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.497164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.497642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.498122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.500648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.502718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.507981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 119/242] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:10.514034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.576021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.578082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.710576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326000-9e65-4cd3-a17d-294c51788b75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.731207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:10.745492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 124/243] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:12:10.751349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:10.772748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:10.788659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.789290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.789570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.789845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.790545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.792011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.836405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 120/244] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:10 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:10.878003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fc68-bd08-4210-8bca-e1ab31038531 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:10.919524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:10.920860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:10.921386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:10.921877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:10.923441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:10.925559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c1b72-1931-403f-ab90-0c37228965f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:10.927715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 125/245] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:10 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:10.937136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.065118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45728c-8dc9-4d7e-a05b-7ed9ae52f9b1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:11.101843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:11.102458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.102739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.103012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.103808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=4949ac25ad1a4e598c4e31b21bd6c141) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:11.104146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:11.105593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a10c-915b-4560-847e-f0b8230e8a9c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:11.111448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 121/246] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:10 2022] GET /identity/v3/role_assignments?user.id=4949ac25ad1a4e598c4e31b21bd6c141&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:11.221879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635459cb-1a75-43e0-89f3-7360eef1943d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.222513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635459cb-1a75-43e0-89f3-7360eef1943d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.223016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635459cb-1a75-43e0-89f3-7360eef1943d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.225890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 126/247] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:11.231365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.231647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.231920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.269505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:11.271541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:11.296843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:11.315755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c872be2-0ce2-4f28-ba95-03078b9b4a8a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.316506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c872be2-0ce2-4f28-ba95-03078b9b4a8a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.317005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c872be2-0ce2-4f28-ba95-03078b9b4a8a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.320044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 127/248] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:11.325557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cbf8-8001-4111-8d52-983aee225c15 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.326101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cbf8-8001-4111-8d52-983aee225c15 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.326573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198cbf8-8001-4111-8d52-983aee225c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.351327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3198cbf8-8001-4111-8d52-983aee225c15 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:11.353666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3198cbf8-8001-4111-8d52-983aee225c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:11.364623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 122/249] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:11.390105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.446811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 128/250] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:11.460760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c0e8-9f29-4dfd-b5ae-7f1d3e6197f2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.461061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c0e8-9f29-4dfd-b5ae-7f1d3e6197f2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.461313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c0e8-9f29-4dfd-b5ae-7f1d3e6197f2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.462967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 129/251] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:11.469122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.469529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.469806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.492627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:11.494373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:11.508712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:11.546523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052bed4-a343-4c70-a2ac-31cab60f33c6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:11.574180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 130/252] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:11.575026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.590310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:11.590964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.591387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.591662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.605945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:11.613113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:11.613985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:11.614794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:11.615835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 123/253] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:11.620932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.647711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:11.699145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27f566-3f40-4d08-b185-8546bb7dc71b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:11.738896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:11.740095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.740372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.740638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.752161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:11.763985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:11.765077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:11.765904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 131/254] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:11.766614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.772423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:11.838218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b118-6837-4f3a-b423-293eb4217428 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:11.887989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5e460-fdef-4313-ab24-4a4ed0d2313a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:11.922815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:11.923431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.923710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.923998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.924715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:11.925010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:11.926497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:11.927786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:11.928608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:11.929028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:11.929442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:11.941098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 132/255] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:11.942263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:11.954633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:11.959766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:11.960652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:11.961406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 124/256] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:11.962041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.018287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.081863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23024930-9342-4219-8530-ebaa12ba8d2c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.103260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.128595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c66ac-5398-4c66-b5a3-6929d4593b96 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.166682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.167600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.168303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.168749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] PATH_INFO: `/v3/users/55f0ea4a51504563a788c1c44a2f02d5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.170086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] RBAC: Authorizing `identity:get_user(user_id=55f0ea4a51504563a788c1c44a2f02d5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.172028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.179275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 125/257] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/users/55f0ea4a51504563a788c1c44a2f02d5 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.180216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.180550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.180982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.181536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.181639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.182417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:12.182790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.184273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.197783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 133/258] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.198710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.295668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4f5ed-908e-4629-a4af-90d5e299acf5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.337731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.338666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.339116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.340919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.341821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:12.343323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.344914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.362354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 126/259] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.366491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.392278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.409513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb62372-85dc-4daa-8ebc-8e3328397f7d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.459371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.460053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.460336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.460605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.461986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.463896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.468299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.470212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 134/260] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.471030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.583693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4771c5-c019-46d6-9165-8fb434751b5f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.618175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.618731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.619008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.619274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.620461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.621953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.628073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 135/261] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.628849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.638768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53887e4-7bdc-41a1-b4e3-9d9757aeca3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.691114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.711108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.712309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.712795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.713267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] PATH_INFO: `/v3/users/18990279906c40b58ee724e89dcb45ca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.717813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] RBAC: Authorizing `identity:get_user(user_id=18990279906c40b58ee724e89dcb45ca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.719495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.721471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 127/262] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/users/18990279906c40b58ee724e89dcb45ca => generated 283 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:12.722499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.742854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f50da-1980-4471-8985-fb3ba1365af2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.783871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.784572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.784882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.785122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/55f0ea4a51504563a788c1c44a2f02d5/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.787132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user_id=55f0ea4a51504563a788c1c44a2f02d5, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:12.791552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:12.815330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 136/263] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:12 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/55f0ea4a51504563a788c1c44a2f02d5/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:12.816063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:12.872479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:12.917220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc8c9-765d-4487-bef7-ede5f0c476ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:12.981288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:12.981921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:12.982223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:12.982525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:12.994222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7dc4e-518c-4052-91c8-8ed6b3d85553 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:13.019938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:13.034567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:13.035201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.035483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.035760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.062513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.064326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.121512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.123097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.222025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:13.229826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 137/264] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:13.236224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:13.288367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:13.319160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 128/265] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:13.324916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:13.347217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4c1b2-5f1a-441e-8a7d-76bf5918e3b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:13.393025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:13.393614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.393901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.394183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.396713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.402448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.406646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 138/266] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:13 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:13.410216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:13.551736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dc6-0d02-4676-a31f-52a8ddc0dabd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:13.587944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968922-7113-4fd3-aa4d-61014394e64f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:13.600345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:13.621428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:13.622090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.622369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.622642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.623727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:13.624374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.624656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.624932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.625824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=18990279906c40b58ee724e89dcb45ca) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:13.626135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.627587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e73b4b-f87e-4d5f-bd80-ed9fa85a0e4b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.628705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.630705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.632691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 129/267] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:13 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:13.634621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 139/268] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:13 2022] GET /identity/v3/role_assignments?user.id=18990279906c40b58ee724e89dcb45ca&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=9a068445d06b48629659d5184c3e91bc => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:13.638459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:13.650452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8b27-ffb7-4a0f-9dce-29fd122e234f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.650748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8b27-ffb7-4a0f-9dce-29fd122e234f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.650930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8b27-ffb7-4a0f-9dce-29fd122e234f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.652898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 140/269] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:13.658064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.658347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.658625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.687208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:13.688444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:13.738764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:13.843427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39add88-a6d7-4716-a9d8-5aa8cc13222d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:13.853787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 141/270] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:13.891152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:13.891821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.892084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.892356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.893160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=18990279906c40b58ee724e89dcb45ca) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:13.893466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:13.895075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afde8b-1471-4127-922b-577bb02c83ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:13.896483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:13.906850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 130/271] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:13 2022] GET /identity/v3/role_assignments?user.id=18990279906c40b58ee724e89dcb45ca&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:13.908003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112fa-1f73-40fc-bc90-83ac55f28367 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.908299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112fa-1f73-40fc-bc90-83ac55f28367 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.908571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112fa-1f73-40fc-bc90-83ac55f28367 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.909940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 131/272] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:13.921417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2855-6b18-4be4-be2d-502f8c5bff98 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:13.921720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2855-6b18-4be4-be2d-502f8c5bff98 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:13.921953 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2855-6b18-4be4-be2d-502f8c5bff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:13.952014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a13f2855-6b18-4be4-be2d-502f8c5bff98 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:13.953265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a13f2855-6b18-4be4-be2d-502f8c5bff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:14.044442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.060452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6da49-1ad1-4e38-ad9f-29ef332ee55e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.100615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.101298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.101624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.101900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.114919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:14.120773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:14.121699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:14.122480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:14.123103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 142/273] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:14.131089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.140654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 132/274] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:14.147978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb00a-aa02-4e64-b385-5a6a4bf55ea7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.148562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb00a-aa02-4e64-b385-5a6a4bf55ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.148986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb00a-aa02-4e64-b385-5a6a4bf55ea7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.150682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 133/275] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:14.156534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.157060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.157495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.181533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:14.182963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:14.216973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.254859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc47c-ce95-4500-aa70-4deb71ef14d5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.280642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 134/276] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:14.281508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.301231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.301883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.302717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.302955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.303684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:14.304004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.306174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.318899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 143/277] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.318899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.329223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.396199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba301d0-d7e8-4911-88d4-c892555366ae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.425638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898da7-c6a6-4058-9a90-31112d31293d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.433088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.433738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.434017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.434293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.444372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:14.449361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:14.450205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:14.451088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 135/278] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:14.451874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.469205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.469850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.470130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.470399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.471591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.473099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.475522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 144/279] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.476374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.483918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.561300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.571490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c22fb-6f39-4669-a714-1d717450db20 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.585671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14a75e-ade3-4d3a-9e7f-8f271559f0a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.608018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.608774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.609059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.609333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] PATH_INFO: `/v3/users/ce05be5f14b14975a13c07d21c19a6f1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.619481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.620128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.620409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.620681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.621389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:14.621684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.623052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.624601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] RBAC: Authorizing `identity:get_user(user_id=ce05be5f14b14975a13c07d21c19a6f1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.626179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.628318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 136/280] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/users/ce05be5f14b14975a13c07d21c19a6f1 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.628904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.636363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 145/281] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.637978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.715689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:14.730899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20266d8b-81ff-4ddd-b09d-9f115f2148f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.751820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f76f-6587-4b17-9b29-e7df14b1bee7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.774290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.774670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.775015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.775292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.776002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.777523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.794191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.794841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.795128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.795670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.796465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, user.id=ce05be5f14b14975a13c07d21c19a6f1, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:14.796798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.798234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.800090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 137/282] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:14 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:14.801265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:14.812245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 146/283] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/role_assignments?user.id=ce05be5f14b14975a13c07d21c19a6f1&scope.system=all&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.940851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9c90-8f19-447c-aa80-2f3654f7e7f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:14.985673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:14.986403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:14.986595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:14.987639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] PATH_INFO: `/v3/users/4949ac25ad1a4e598c4e31b21bd6c141` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:14.988337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] RBAC: Authorizing `identity:get_user(user_id=4949ac25ad1a4e598c4e31b21bd6c141)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:14.991112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:14.992154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 138/284] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/users/4949ac25ad1a4e598c4e31b21bd6c141 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:14.997013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:15.001213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:15.112896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2b7d9-866a-4987-bbab-505823bb5b7d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:15.155693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:15.156377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:15.156661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:15.156940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:15.158119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:15.159568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125efc25-d49b-4e73-ad71-1fa235654c9c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:15.161313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 147/285] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:14 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:15.167153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:15.184620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:15.285791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ffde8-2a56-430b-a14a-2350f21b24c9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:15.322154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:15.322824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:15.323100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:15.323370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/4949ac25ad1a4e598c4e31b21bd6c141/roles/d28acb0ee63a4d7d87658a048960d8fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:15.325348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user_id=4949ac25ad1a4e598c4e31b21bd6c141, role_id=d28acb0ee63a4d7d87658a048960d8fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:15.326786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354636d-39c4-47b1-8aec-ddeaf5d30fb4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:15.342898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 139/286] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:15 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/users/4949ac25ad1a4e598c4e31b21bd6c141/roles/d28acb0ee63a4d7d87658a048960d8fb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:15.509124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809070-bb0c-4575-9288-d59bafccebd7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:15.509651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809070-bb0c-4575-9288-d59bafccebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:15.510054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809070-bb0c-4575-9288-d59bafccebd7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:15.511884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 148/287] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:15.517859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:15.518145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:15.518420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:15.540173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:15.541438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:15.613449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:15.622073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 140/288] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:15.646339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809070-bb0c-4575-9288-d59bafccebd7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:15.743754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:15.793346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809070-bb0c-4575-9288-d59bafccebd7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:15.843566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:15.844246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:15.844529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:15.844803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:15.858646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:15.864978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:15.865843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:15.866604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 149/289] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:15.873119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:15.980325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8678451-47aa-4fa1-b895-e2010718f1bb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:16.019470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:16.020123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:16.020404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:16.020754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:16.021523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:16.022346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:16.023734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:16.032496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 141/290] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:16.038098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:16.165825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695961f4-c2f2-481a-a32c-9a0e591bc8c7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:16.196033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:16.202180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:16.202828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:16.203166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:16.203442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] PATH_INFO: `/v3/users/55f0ea4a51504563a788c1c44a2f02d5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:16.204699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] RBAC: Authorizing `identity:get_user(user_id=55f0ea4a51504563a788c1c44a2f02d5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:16.206190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:16.211637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 150/291] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:16 2022] GET /identity/v3/users/55f0ea4a51504563a788c1c44a2f02d5 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:16.214168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:16.288433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:16.319436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e311860-5d1d-4407-a053-bef0acd06654 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:16.353881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:16.354498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:16.354784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:16.355054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:16.426346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:16.428611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:16.490289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:16.531201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 142/292] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:16.537486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:16.639908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73495-3e62-4fd9-b3b5-56a9791f7a7f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:16.674116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:16.674753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:16.675032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:16.675306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:16.676603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:16.678849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c302ae-0818-4ed6-9db6-12aa3ab739df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:16.680768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 151/293] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:16 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:16.686545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:16.788456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2388db-f9f9-4197-aa08-a8363147b93c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:16.837996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:16.838610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:16.838884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:16.839154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:16.839937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=55f0ea4a51504563a788c1c44a2f02d5) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:16.840284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:16.841715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59bbb8-8d6c-469f-96e5-dcf48f74ee66 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:16.848444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 143/294] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:16 2022] GET /identity/v3/role_assignments?user.id=55f0ea4a51504563a788c1c44a2f02d5&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:17.271390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ac45-7e95-4e9d-9202-ce2e57a611f9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.271883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ac45-7e95-4e9d-9202-ce2e57a611f9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.272117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ac45-7e95-4e9d-9202-ce2e57a611f9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.273578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 152/295] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:17.278917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1adb-8a35-436d-b218-63113fb92402 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.279312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1adb-8a35-436d-b218-63113fb92402 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.279491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1adb-8a35-436d-b218-63113fb92402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.299562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-76cb1adb-8a35-436d-b218-63113fb92402 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:17.300783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76cb1adb-8a35-436d-b218-63113fb92402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:17.323099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:17.372622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 144/296] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:17.394633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8ac45-7e95-4e9d-9202-ce2e57a611f9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.469294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:17.502904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8ac45-7e95-4e9d-9202-ce2e57a611f9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:17.534153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935909b-8fda-4368-be91-6bcccb5853ec None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.534491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935909b-8fda-4368-be91-6bcccb5853ec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.534716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935909b-8fda-4368-be91-6bcccb5853ec None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.537147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 145/297] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:17.540809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:17.541460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.541738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.541985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.542084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.542276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.542556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.554921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:17.560940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:17.561245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:17.563262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:17.564865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:17.565631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:17.565714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 153/298] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:17.568873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.638055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 146/299] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:17.662489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.699110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de603f5-a22b-414e-9521-df96d078d430 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:17.738497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:17.748895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:17.749516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.749831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.750107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.750822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:17.751120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:17.752504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84fa79-2258-41ac-82aa-bde31ad6fe54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:17.760305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:17.765812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 154/300] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:17.768296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c344-4a1e-4620-b353-84e75ed3bb14 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.768572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c344-4a1e-4620-b353-84e75ed3bb14 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.769037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c344-4a1e-4620-b353-84e75ed3bb14 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.771109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 155/301] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:17.771803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02c344-4a1e-4620-b353-84e75ed3bb14 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.782569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434b2de-32c7-474a-8f8b-72ccc0643382 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:17.818267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:17.818911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.819190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.819459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.829677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-98c1466d-4339-4c81-b39f-1e34a7d84cef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:17.835249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:17.836146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:17.836922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 147/302] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:17.840103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.840567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.841649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.859392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:17.860616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:17.887634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02c344-4a1e-4620-b353-84e75ed3bb14 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:17.921009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:17.921642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.921926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.922197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] PATH_INFO: `/v3/users/ce05be5f14b14975a13c07d21c19a6f1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.923350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] RBAC: Authorizing `identity:get_user(user_id=ce05be5f14b14975a13c07d21c19a6f1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:17.924819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d598a3-853b-455e-83db-4f0fd5951068 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:17.926017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 148/303] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:17.930975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 156/304] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:17 2022] GET /identity/v3/users/ce05be5f14b14975a13c07d21c19a6f1 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:17.931116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.931192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dd68b-fb0f-4326-a278-57f19f2642f0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:17.931192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dd68b-fb0f-4326-a278-57f19f2642f0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:17.931192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dd68b-fb0f-4326-a278-57f19f2642f0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:17.931192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 157/305] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:17.934097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dd68b-fb0f-4326-a278-57f19f2642f0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:17.991199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.011707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.034630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691940e-1bba-4af2-84d5-3d62de53adc2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.037441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dd68b-fb0f-4326-a278-57f19f2642f0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.069462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.070072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.070352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.070618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.070997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.071322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:18.071619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.071705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.071919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.072224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] PATH_INFO: `/v3/system/users/ce05be5f14b14975a13c07d21c19a6f1/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.073027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb13a1-6f68-49f3-adfe-b491d95a073b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.073675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ce05be5f14b14975a13c07d21c19a6f1, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.075046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.081505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 149/306] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:18.082614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.082877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.083839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.088489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 158/307] 10.222.0.72 () {58 vars in 1334 bytes} [Thu Aug 4 10:12:17 2022] PUT /identity/v3/system/users/ce05be5f14b14975a13c07d21c19a6f1/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:12:18.089309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.112486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:18.113828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:18.188933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 150/308] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:18.190032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.224231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d772-026f-4058-a022-5575cc81ece2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.247154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.267431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.268018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.268339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.268559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.269960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.271892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.273703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 159/309] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:18.274514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.274991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.312941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c398268-4bbd-4bf8-8581-f0facfbaed60 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.353732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.354376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.354657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.354931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] PATH_INFO: `/v3/users/0efee2b6113b40839963671e48d4f8e6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.372237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] RBAC: Authorizing `identity:get_user(user_id=0efee2b6113b40839963671e48d4f8e6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.374394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.376552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 151/310] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/users/0efee2b6113b40839963671e48d4f8e6 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:18.377612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.384773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6793-3af1-41c9-971e-608a13f4f6f1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.421312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.422127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.422259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.422497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.434403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:18.440343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:18.441224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:18.442005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 160/311] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:18.442760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.495667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc531-13ca-4ad7-b009-8221a3009607 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.498129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.524954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.534827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.535452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.535727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.536018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] PATH_INFO: `/v3/groups` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.537314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.538734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.552927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a809e-4bd5-415d-8f08-4b1502ebeaae None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.560409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 152/312] 10.222.0.72 () {60 vars in 1134 bytes} [Thu Aug 4 10:12:18 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:18.560409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.620103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.620990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.621411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.621821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.622927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, user.id=0efee2b6113b40839963671e48d4f8e6, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:18.623384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.625214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.635060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 161/313] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/role_assignments?user.id=0efee2b6113b40839963671e48d4f8e6&scope.system=all&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:18.706813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa808f-3424-4d1b-80a2-e4fd7de682fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.746737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.747377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.747714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.748031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.748781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:18.749086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.750536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.761109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 153/314] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:18.766451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.882097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef583b7f-ac69-4c93-9bc1-a8607e5deaa3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:18.895167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:18.926699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:18.928470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:18.928934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:18.929392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] PATH_INFO: `/v3/users/4949ac25ad1a4e598c4e31b21bd6c141` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:18.931380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] RBAC: Authorizing `identity:get_user(user_id=4949ac25ad1a4e598c4e31b21bd6c141)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:18.933300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:18.935328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 162/315] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/users/4949ac25ad1a4e598c4e31b21bd6c141 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:18.941604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:18.996181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:19.049278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e9ca-1ca4-4d4b-81e9-9583a9725f85 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:19.083910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:19.084573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:19.084858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:19.085133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:19.154952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:19.156460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:19.205783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:19.263016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 154/316] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:12:19.268445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:19.369358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6601-78e1-4708-bde6-dbe1c2f2745b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:19.404008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:19.404896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:19.405384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:19.405809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:19.407131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:19.409044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ea702-dbe8-4243-bc68-6ddf2ec7254d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:19.411426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 163/317] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:19 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:19.417027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:19.519712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29a64a-692b-42bf-a265-d6999694f5de None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:19.554956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:19.555573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:19.555853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:19.556155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:19.556945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, user.id=4949ac25ad1a4e598c4e31b21bd6c141) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:19.557252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:19.558760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a3a3c-c3f7-4d33-9d6e-d5111676cdf3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:19.565006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 155/318] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:19 2022] GET /identity/v3/role_assignments?user.id=4949ac25ad1a4e598c4e31b21bd6c141&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:20.144611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972a33-89ee-48c5-a750-465860f3de7f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.145043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972a33-89ee-48c5-a750-465860f3de7f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.145233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972a33-89ee-48c5-a750-465860f3de7f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.147125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 164/319] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:20.152348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.152637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.152913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.171718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:20.172928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:20.180949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:20.248084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 156/320] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:20.271421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61972a33-89ee-48c5-a750-465860f3de7f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:20.325780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:20.378882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61972a33-89ee-48c5-a750-465860f3de7f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:20.412170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:20.412743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.413036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.413294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.424021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:20.429553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:20.430397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:20.431197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 165/321] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:20.436811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:20.537453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104780-a63f-416f-86a1-9721c39c17f8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:20.570711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:20.571325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.571604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.571876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.572620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:20.572924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:20.574283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dfa89-d052-41fc-a7b1-1d9e7467e90a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:20.586716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 157/322] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:20.588052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:20.688177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388caff-d4fd-4fd8-8c92-348207a08838 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:20.697455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:20.722804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:20.723498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.723806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.724109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] PATH_INFO: `/v3/users/ce05be5f14b14975a13c07d21c19a6f1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.725273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] RBAC: Authorizing `identity:get_user(user_id=ce05be5f14b14975a13c07d21c19a6f1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:20.726709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:20.728492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 166/323] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/users/ce05be5f14b14975a13c07d21c19a6f1 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:20.733239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76d4c9-c58f-43fc-9097-3aba609da405 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.734514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76d4c9-c58f-43fc-9097-3aba609da405 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.734514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76d4c9-c58f-43fc-9097-3aba609da405 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.734695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:20.735311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 158/324] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:20.740514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472682-b6f0-4e84-9ba0-f0cc7c1b81a0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.740804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472682-b6f0-4e84-9ba0-f0cc7c1b81a0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.741084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472682-b6f0-4e84-9ba0-f0cc7c1b81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.748272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:20.761411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9a472682-b6f0-4e84-9ba0-f0cc7c1b81a0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:20.762595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9a472682-b6f0-4e84-9ba0-f0cc7c1b81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:20.829021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 159/325] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:20.830489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514f69-08c9-47b2-9e1f-4cc326a9980d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.830775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514f69-08c9-47b2-9e1f-4cc326a9980d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.831051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514f69-08c9-47b2-9e1f-4cc326a9980d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.836315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 160/326] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:20.838187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.838471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.838747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.839135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7b52-a088-4516-b79f-289ed61d2f31 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:20.856269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:20.857466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:20.877101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:20.877712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:20.878025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:20.878294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:20.879079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] RBAC: Adding query filter params (role.id=ed22ce3e92574f48b1aaa82fa1077beb, user.id=ce05be5f14b14975a13c07d21c19a6f1, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:20.879383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:20.881388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:20.892620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 167/327] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/role_assignments?user.id=ce05be5f14b14975a13c07d21c19a6f1&scope.system=all&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:20.893385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:20.935827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 161/328] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:20.944201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:20.958463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:21.013356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dd689-6052-4532-8382-4fcc5d99e999 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.031833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:21.050871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.051527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.051863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.052919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.054133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:21.055578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:21.057194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 168/329] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:21.063358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:21.085173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0807e-7fcd-4ae1-b754-d276001a02ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.124704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.125344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.125628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.125900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.135562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:21.140508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:21.141563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:21.142545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 162/330] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:21.147479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:21.172864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c352e7-fa88-4179-abd4-3d1440d745d2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.174344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:21.206430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.206970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.207255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.207530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] PATH_INFO: `/v3/groups` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.208252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:21.209700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:21.226789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 169/331] 10.222.0.72 () {60 vars in 1134 bytes} [Thu Aug 4 10:12:21 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:12:21.249468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036675-5f51-4f27-9058-5e8c853d4404 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.275525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:21.287982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.288640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.288946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.289222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.289977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:21.290272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:21.291708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:21.301653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 163/332] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:21.308172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:21.410776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bd0a3-c7cb-4acc-92b6-ffde631f1861 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.444399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.445040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.445323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.445598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] PATH_INFO: `/v3/users/0efee2b6113b40839963671e48d4f8e6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.446858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] RBAC: Authorizing `identity:get_user(user_id=0efee2b6113b40839963671e48d4f8e6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:21.448408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d49432-def6-4511-ba90-d378ba04556f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:21.450311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 170/333] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:21 2022] GET /identity/v3/users/0efee2b6113b40839963671e48d4f8e6 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:21.455747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:21.565462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519023-bd7e-42be-90f0-6e86c58c6b16 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:21.601739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:21.602415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:21.602702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:21.603036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] PATH_INFO: `/v3/system/users/0efee2b6113b40839963671e48d4f8e6/roles/d28acb0ee63a4d7d87658a048960d8fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:21.604674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0efee2b6113b40839963671e48d4f8e6, role_id=d28acb0ee63a4d7d87658a048960d8fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:21.606041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cd96b-ab84-4cfe-a4c8-99138ec3bbe8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:21.616693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 164/334] 10.222.0.72 () {58 vars in 1334 bytes} [Thu Aug 4 10:12:21 2022] PUT /identity/v3/system/users/0efee2b6113b40839963671e48d4f8e6/roles/d28acb0ee63a4d7d87658a048960d8fb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:12:23.706660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eec9-740a-4827-acde-62aa01431029 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:23.707750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eec9-740a-4827-acde-62aa01431029 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:23.708299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eec9-740a-4827-acde-62aa01431029 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:23.710106 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 171/335] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:23.716433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3bab-ca7e-4c65-a200-518648433a65 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:23.717288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3bab-ca7e-4c65-a200-518648433a65 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:23.717761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3bab-ca7e-4c65-a200-518648433a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:23.741191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c8bc3bab-ca7e-4c65-a200-518648433a65 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:23.742698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c8bc3bab-ca7e-4c65-a200-518648433a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:23.804510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:23.830271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 165/336] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:23.870143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742eec9-740a-4827-acde-62aa01431029 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:23.907055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:23.956357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1bb7-118c-4f90-8a5b-9e7dba3802dc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:23.956735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1bb7-118c-4f90-8a5b-9e7dba3802dc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:23.956900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1bb7-118c-4f90-8a5b-9e7dba3802dc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:23.958298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 166/337] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:23.963746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:23.964064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:23.964346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.006007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:24.007209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:24.012639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742eec9-740a-4827-acde-62aa01431029 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.069263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.069922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.070206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.070476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.093336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:24.099400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:24.100363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:24.101112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:24.109208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 172/338] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:24.113130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 167/339] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:24.116060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.153893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.224480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:24.238707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922cabb-9ced-4dc9-bfa8-af2e2a85109a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.330958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.331591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.331868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.332228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.333005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:24.333811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:24.335191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64abd44-babe-4b3b-97e1-b151d7df15d1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:24.343987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 173/340] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:24.349649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7045d-cea2-41e3-bb36-f7c4a0312f04 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.349936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7045d-cea2-41e3-bb36-f7c4a0312f04 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.350212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7045d-cea2-41e3-bb36-f7c4a0312f04 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.351506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 174/341] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:24.353536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7045d-cea2-41e3-bb36-f7c4a0312f04 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.375218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e1ada-5999-4e84-83b9-eac34dc77476 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.396576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:24.424200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.424841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.425116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.425440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.438970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4b0c102d-eed8-44c7-9c07-649f32c73935 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:24.445354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:24.446190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:24.446948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:24.452789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 168/342] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:24.454209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.454491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.454762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.477547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:24.478664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:24.519400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7045d-cea2-41e3-bb36-f7c4a0312f04 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.559849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.560578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.560859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.561133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.568107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 169/343] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:24.568107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.604235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:24.665384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:24.667140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:24.694219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6efbe8-fc18-4e3e-ae86-e53cb90ecbec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.696288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:24.735443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.736874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.737355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.737823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.738755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:24.739290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:24.740895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781e33-22af-462c-bb3e-746bdf46f149 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:24.749944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 170/344] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:24.751549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc9e5-477f-41ed-8f43-fccf2506da9c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.752059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc9e5-477f-41ed-8f43-fccf2506da9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.752528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc9e5-477f-41ed-8f43-fccf2506da9c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.754020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 171/345] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:24.755529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dc9e5-477f-41ed-8f43-fccf2506da9c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.843838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 175/346] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:12:24.845731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:24.880314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dc9e5-477f-41ed-8f43-fccf2506da9c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:24.922826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:24.923509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.923788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.924090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.935817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:24.941213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8920b4fa-9a5f-46af-a701-238c08625746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:24.944729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:24.945667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:24.946449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:24.949204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 172/347] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:24.950602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:24.951016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:24.951415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:24.973257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:24.974856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:24.994257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.008339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751682-5df6-4ffe-93b7-2ef11d4afb5e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.059250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.059820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.060176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.060450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.072542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 173/348] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:25.073720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.179849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.181984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.193724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89fe6d-19b0-413f-9585-d8485fc70cef None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.234746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.235527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.239084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.242275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.244559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.247685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.251694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 174/349] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:25.253201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.267688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.282818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.322613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 176/350] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:25.323656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.379371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac726-0764-460c-8e23-c2cf3ef112db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.419577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db704b0-09a2-4600-a084-01331364111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.420223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db704b0-09a2-4600-a084-01331364111e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.420506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db704b0-09a2-4600-a084-01331364111e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.421037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db704b0-09a2-4600-a084-01331364111e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.421759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db704b0-09a2-4600-a084-01331364111e None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:25.422064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db704b0-09a2-4600-a084-01331364111e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.423448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db704b0-09a2-4600-a084-01331364111e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.432563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 175/351] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:25.433796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451bdae-2c89-4bf2-a5e8-d3b68b5c87d8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.434087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451bdae-2c89-4bf2-a5e8-d3b68b5c87d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.434360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451bdae-2c89-4bf2-a5e8-d3b68b5c87d8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.435944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 176/352] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:25.436667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451bdae-2c89-4bf2-a5e8-d3b68b5c87d8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.437638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeba55-ff93-49eb-ae37-7454cb0de750 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.480739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.481743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.482025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.482295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.500591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:25.508181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:25.509089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:25.509873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 177/353] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:25.510566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.531807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.544160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.580589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451bdae-2c89-4bf2-a5e8-d3b68b5c87d8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.622030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.622631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.622912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.623186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.635052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.637241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.638499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4de38b-2c63-46ef-b801-5f582b4ca78d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.639193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 177/354] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:25.639738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.688668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.689317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.689595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.689864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.691082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.693098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8124f-9108-4243-b628-8905efcf1dbb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.695076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 178/355] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:25.696278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.696558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.696835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.716799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:25.718751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:25.769987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c323f2-36d2-4e7b-af44-41f33810c418 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.805754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 179/356] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:25.806727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:25.811359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.812259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.812714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.813155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.821387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.836057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:25.905236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.906738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.933904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bebc-a0dc-40e2-a5c6-b01de49d6225 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:25.976763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:25.977380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:25.977661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:25.977932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:25.978937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:25.979237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:25.980697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:25.990219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 180/357] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:25.991024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.017623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 178/358] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:26.023622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.074186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.079080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.113529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbe60-0f0f-455b-9eda-272c5130b6d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.140868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ec159-3133-4408-a587-9435e3bd54f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.156374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.157239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.157524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.157800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.158603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:26.158915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.160411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.167718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 181/359] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:25 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.168415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.182431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.183111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.183386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.183657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.186073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.187717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5eeb7-8673-4088-bb19-f25dbcde7fcd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.189664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 179/360] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.191995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54502-651f-4dba-9b54-a4b8a8b86441 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.191995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54502-651f-4dba-9b54-a4b8a8b86441 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.192185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54502-651f-4dba-9b54-a4b8a8b86441 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.193581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 180/361] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:26.195436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc54502-651f-4dba-9b54-a4b8a8b86441 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.325289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fabb4-0e91-493b-92e6-4379b423dac1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.337142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc54502-651f-4dba-9b54-a4b8a8b86441 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.356435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.369747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.370746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.371026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.371292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.374333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.385703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.386352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.386697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.386981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] PATH_INFO: `/v3/users/0efee2b6113b40839963671e48d4f8e6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.388282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:26.388484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] RBAC: Authorizing `identity:get_user(user_id=0efee2b6113b40839963671e48d4f8e6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.389903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.391515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 181/362] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/users/0efee2b6113b40839963671e48d4f8e6 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.392289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:26.394992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:26.396136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:26.396939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:26.399010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 182/363] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:26.399921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.521110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f32b4-1934-47e5-bc67-d55d521beafd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.533901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81588b9-8a80-4bc9-9e0f-df74628abd3b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.572844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.573825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.574252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.574525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.576428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.577690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9955d-1d19-4f24-9b06-926a362d6d20 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.579583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 182/364] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.580956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.581100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.581100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.581239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.581311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.581383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.581448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.582235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:26.582541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.587994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.591155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 183/365] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=9a068445d06b48629659d5184c3e91bc => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.592212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.611455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:26.613019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:26.664950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.684609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.717169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 183/366] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:26.719860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3338d-57fd-4c1a-9fe7-2a079c2ce310 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.724369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.777094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.778143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.778630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.779129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.780190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] RBAC: Adding query filter params (role.id=d28acb0ee63a4d7d87658a048960d8fb, user.id=0efee2b6113b40839963671e48d4f8e6, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:26.780501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.782028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.798357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 184/367] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/role_assignments?user.id=0efee2b6113b40839963671e48d4f8e6&scope.system=all&role.id=d28acb0ee63a4d7d87658a048960d8fb => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.799091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.875873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b22e-2d01-4bb6-b191-057a8850c883 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.928882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.929509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.929784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.930052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.930776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:26.931074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.932473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.934252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ed9b-d526-43f5-92ce-087d01b8c283 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:26.948656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 184/368] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.949448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:26.956594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:26.982773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:26.983938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:26.984598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:26.985039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:26.986868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:26.988935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:26.991808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 185/369] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:26.992825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.005700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.078650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795a56-04a8-4dad-b4a4-50b0ae792d44 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.113676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d26d-0c7e-45ce-bfd4-51348fc43b63 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.115524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.116248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.116525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.116795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.129632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:27.136348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:27.137306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:27.138113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 185/370] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:27.138799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.151511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.152184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.152465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.152739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.212705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.229174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.230649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.258006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da4ed-9d01-4abc-ac2b-a327d8470418 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.260719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.298995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.299755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.300157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.300525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.303292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:27.303620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.306144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.312244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 186/371] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:27.317092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.356248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 186/372] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:27.365862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.495585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e500e2-fdc4-4019-a4d7-457e77d77d90 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.544861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.546193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.546692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.547169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.548180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:27.548693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.550833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.561044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.565460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 187/373] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:27.571398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.580539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47759-664b-411f-9a46-e5d98d353dad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.613233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.629346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.630556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.630832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.631098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.634514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.635949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.642726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 187/374] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:27.644212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.717280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579631-f0c8-4b9b-b385-b823f8d1d832 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.754310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc929-f12d-403c-969d-54edacc5f629 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.777432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.778093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.778315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.778581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.792361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.793532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.794006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.794479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.796272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.797918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.799913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 188/375] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:27.806651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:27.833939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.891472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.893019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.916088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51755b-9052-4794-a163-b20c33d40a6e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:27.942545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:27.955057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:27.956311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:27.956789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:27.957335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:27.958353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:27.958856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:27.960547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:27.967032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 189/376] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=9a068445d06b48629659d5184c3e91bc => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:28.055903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 188/377] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:28.061639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:28.171029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:28.188879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720464f-c6d4-451b-b8c9-0a131b87fe35 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:28.224414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:28.225350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:28.225836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:28.226301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] PATH_INFO: `/v3/projects/8d75901b5a184df7b4b0c986ca5abc96` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:28.227648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d75901b5a184df7b4b0c986ca5abc96)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:28.229313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:28.231463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 190/378] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:28 2022] GET /identity/v3/projects/8d75901b5a184df7b4b0c986ca5abc96 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:28.237741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:28.360104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d20b2-d205-41dd-8de5-ab234a7e64d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:28.396878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:28.409556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:28.410391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:28.410675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:28.410970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] PATH_INFO: `/v3/groups/c765bb79131c464e8758e806df48844c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:28.423679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] RBAC: Authorizing `identity:get_group(group_id=c765bb79131c464e8758e806df48844c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:28.425166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc945407-53de-4d7f-ac79-5b48fa69dae2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:28.426828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 189/379] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:28 2022] GET /identity/v3/groups/c765bb79131c464e8758e806df48844c => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:28.432764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:28.544746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc078a3-c797-4199-8daa-94bc6ef90bd8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:28.579562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:28.580215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:28.580492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:28.580763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:28.581556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] RBAC: Adding query filter params (group.id=c765bb79131c464e8758e806df48844c, role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=8d75901b5a184df7b4b0c986ca5abc96) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:28.582028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:28.583908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98fce-0fdd-4a9f-bbe7-3a580abc5067 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:28.590299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 191/380] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:28 2022] GET /identity/v3/role_assignments?group.id=c765bb79131c464e8758e806df48844c&scope.project.id=8d75901b5a184df7b4b0c986ca5abc96&role.id=abf8a86280c54a0798604e6c88e690cf => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:29.015135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b5cf-0a7d-4584-a8d4-c2d9cc46ba4d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.016185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b5cf-0a7d-4584-a8d4-c2d9cc46ba4d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.016676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b5cf-0a7d-4584-a8d4-c2d9cc46ba4d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.018219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 190/381] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:29.025061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96903aeb-fdd9-47cc-9072-b0945c996e82 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.025935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96903aeb-fdd9-47cc-9072-b0945c996e82 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.026418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96903aeb-fdd9-47cc-9072-b0945c996e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.047708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-96903aeb-fdd9-47cc-9072-b0945c996e82 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:29.049231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-96903aeb-fdd9-47cc-9072-b0945c996e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:29.063347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:29.128080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 192/382] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:29.148929 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b5cf-0a7d-4584-a8d4-c2d9cc46ba4d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.189775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684edef-44be-46a9-93e0-67d403841c8a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.190137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684edef-44be-46a9-93e0-67d403841c8a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.190320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684edef-44be-46a9-93e0-67d403841c8a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.191669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 193/383] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:29.199178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.199459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.199727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.232304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:29.233491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:29.299109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b5cf-0a7d-4584-a8d4-c2d9cc46ba4d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:29.360037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 194/384] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:29.365607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:29.366261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.366537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.366808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.384485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.388487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:29.393472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:29.394436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:29.395748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 191/385] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:29.401783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.475549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:29.528313 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da02b7d-0b23-47b4-bd3f-19583483075b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:29.544013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:29.571517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:29.572574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.572864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.573138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.573872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:29.574179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:29.575547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:29.587333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecead4-6c43-4ca5-9286-97965fc11a28 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:29.598312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 192/386] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:29.604246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.671215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:29.671882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.672187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.672461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.701434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:29.706481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:29.707484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:29.708807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 195/387] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:29.715737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.728489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b3ac-0fb9-4df9-9570-c61df3e28ad5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:29.744092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:29.768116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:29.768762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.769041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.769313 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.771042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:29.772522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:29.774692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 193/388] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:29.780160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.896177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000281d-1ca5-4214-b98f-d63d29e65d0c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:29.935088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:29.935742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:29.936055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:29.936760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:29.937955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:29.939764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:29.941453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 194/389] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:29.948012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:29.977370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0651c-7e05-4ce0-977f-61e772ee8e01 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.016966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.018143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.018849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.019291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.019565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.020903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:30.021220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.023906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b44dd6-dd96-40be-99cd-c6c7731a0475 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.030223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.038431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 196/390] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:30.039683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a439b7-4648-49cf-bca7-9fe865dfa65c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.039986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a439b7-4648-49cf-bca7-9fe865dfa65c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.040264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a439b7-4648-49cf-bca7-9fe865dfa65c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.041518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 197/391] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:30.042980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a439b7-4648-49cf-bca7-9fe865dfa65c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.089133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef47df-dc4d-4b14-8480-131165f899ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.133502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.134281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.135078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.135418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/groups/476d317832c0447995cd8c7448193f6e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.138207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, group_id=476d317832c0447995cd8c7448193f6e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.140465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbabfa-c5fd-49a1-90f3-d93126f8d6f1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.159782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 195/392] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Aug 4 10:12:29 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/groups/476d317832c0447995cd8c7448193f6e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:12:30.162281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.162629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.162938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.172459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a439b7-4648-49cf-bca7-9fe865dfa65c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.203968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:30.205210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:30.256891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.258456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.258738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.259012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.262145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.269136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.271647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 198/393] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:30.281712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.332061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 196/394] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:30.373075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.394837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.421401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.435083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e41c-58c2-4390-aa7c-6c53bbb9816d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.470588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.471593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.472097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.472754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.474662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.476311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.478550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 199/395] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:30.486692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.498402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344ea11-34aa-42e2-a3d8-34d9d0ffcdf3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.543005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.544450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.545029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.545472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.559033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:30.565771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:30.566852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:30.567636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:30.576566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 197/396] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:30.584653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.616793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645508bf-7195-4dcf-b8eb-e82df61ed875 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.655576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.656651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.657165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.657629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/groups/476d317832c0447995cd8c7448193f6e/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.660652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00df1bf34b24986ab49f47aa7d3962c, group_id=476d317832c0447995cd8c7448193f6e, role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.663110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a7546-20b9-42c6-81c2-3ab57f7ede59 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.667270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.683277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 200/397] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Aug 4 10:12:30 2022] PUT /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c/groups/476d317832c0447995cd8c7448193f6e/roles/9a068445d06b48629659d5184c3e91bc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:30.724173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ce86-046f-493f-a32f-8938f7b11633 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.725079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ce86-046f-493f-a32f-8938f7b11633 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.725564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ce86-046f-493f-a32f-8938f7b11633 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.732937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 201/398] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:30.740780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.741397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.741710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.778251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:30.779511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:30.788242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:30.796309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461b2e-7ce7-4694-a23b-090807738318 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:30.866533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:30.867168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:30.867446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:30.867714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:30.868473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:30.868774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:30.870137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:30.883408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 198/399] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:30.894748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:30.902822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 202/400] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:30.936007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.032103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82181a-1a57-4461-a126-e1fb1e23eb71 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.071823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.072490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.072774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.073049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.080732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.082389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.089174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 199/401] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:31.102285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.123892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3c0c-7fb5-48a8-9e3b-e7b723ab3ee0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.159437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.167511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.168480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.168985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.169539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.186787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.193899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-617dc095-e3d3-4e2f-a1c5-af1bb53a0270 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:31.205572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:31.206471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:31.207282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:31.210685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 203/402] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:31.212283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c9f-4482-4dda-b931-53817dfd8dcd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.215980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c9f-4482-4dda-b931-53817dfd8dcd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.216266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c9f-4482-4dda-b931-53817dfd8dcd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.217828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 204/403] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:31.218545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e35c9f-4482-4dda-b931-53817dfd8dcd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.305821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c538ff4-03cd-498b-8eba-45751cfd0289 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.344571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e35c9f-4482-4dda-b931-53817dfd8dcd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.346423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.348180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.349165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.349633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.351017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.352652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ff7b1-07db-45f9-98ad-815e007a6f2a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.355033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 200/404] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:31.362350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.363088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.363655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.386740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:31.388165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.388574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:31.388960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.389326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.389668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.390445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:31.390751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.393323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.402941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 205/405] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:31.403348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.456237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.489224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 201/406] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:31.491068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.513572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc007-09d6-4d68-a5ab-fe5150c228dd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.529323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.553751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.554535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.554890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.555238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/groups/476d317832c0447995cd8c7448193f6e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.557504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, group_id=476d317832c0447995cd8c7448193f6e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.559044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.579979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 206/407] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Aug 4 10:12:31 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/groups/476d317832c0447995cd8c7448193f6e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:31.579979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.669232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de949c-1cba-4c14-a9ea-b477d948153b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.740083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.740670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.740941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.741210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.742735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.745089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.747145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 202/408] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:31.752050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.767785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a6f6-7db7-47be-804f-103656837047 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.778094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.811283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.811940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.812254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.812527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.825361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:31.830848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:31.831743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:31.833704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 207/409] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:31.838173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:31.918483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:31.930794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3466d-4fb4-4731-8515-05c08cfa3985 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.948454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5fe94-1932-444b-9547-6697a7b90d0f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:31.983498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:31.984526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:31.985014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:31.985510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:31.986460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:31.987010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:31.988695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:31.998977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 208/410] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:32.006688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:32.013089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.013744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.014043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.014309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.017342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:32.018108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:32.023425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 203/411] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:31 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:32.033070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:32.073898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:32.158584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705914d-b09d-48e7-8bf1-b06cc714925d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:32.197710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.198358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.198642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.198981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] PATH_INFO: `/v3/groups/c765bb79131c464e8758e806df48844c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.200212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] RBAC: Authorizing `identity:get_group(group_id=c765bb79131c464e8758e806df48844c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:32.202646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:32.204572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 209/412] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:32 2022] GET /identity/v3/groups/c765bb79131c464e8758e806df48844c => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:32.211300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:32.242936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d686-f628-4482-ab35-25464a7c0753 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:32.280781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.281462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.281747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.282022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/groups/476d317832c0447995cd8c7448193f6e/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.284256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb, group_id=476d317832c0447995cd8c7448193f6e, role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:32.285689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:32.300498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 204/413] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Aug 4 10:12:32 2022] PUT /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb/groups/476d317832c0447995cd8c7448193f6e/roles/9a068445d06b48629659d5184c3e91bc => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:32.393891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a0afb-8452-4b8c-bd7b-1c2b31f7cc70 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:32.409497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:32.447592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.448287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.448569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.448840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] PATH_INFO: `/v3/projects/8d75901b5a184df7b4b0c986ca5abc96` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.452779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] RBAC: Authorizing `identity:get_project(project_id=8d75901b5a184df7b4b0c986ca5abc96)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:32.454281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffdf4-22c7-4a56-add3-b84cd63c9c15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:32.456475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 210/414] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:32 2022] GET /identity/v3/projects/8d75901b5a184df7b4b0c986ca5abc96 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:32.463073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:32.495740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:32.533055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00186-7b83-47f3-8867-678446a16186 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.533367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00186-7b83-47f3-8867-678446a16186 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.533542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00186-7b83-47f3-8867-678446a16186 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.535116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 211/415] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:32.543084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.543657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.544172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.570361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:32.572306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:32.587987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4983f-4610-4f39-9093-b9b44f36ec99 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:32.636703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.637974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.638261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.638537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] PATH_INFO: `/v3/projects/8d75901b5a184df7b4b0c986ca5abc96/groups/c765bb79131c464e8758e806df48844c/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.642517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d75901b5a184df7b4b0c986ca5abc96, group_id=c765bb79131c464e8758e806df48844c, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:32.646463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:32.670404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 205/416] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Aug 4 10:12:32 2022] PUT /identity/v3/projects/8d75901b5a184df7b4b0c986ca5abc96/groups/c765bb79131c464e8758e806df48844c/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:32.685888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 212/417] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:32.719340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:32.824514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:32.845576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df1d8-5625-4105-85e0-931998f92522 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:32.883212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:32.883915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:32.884228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:32.884505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:32.895106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-835b29ee-5993-48ac-915a-7fb1e528a1c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:32.900202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:32.901079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:32.901862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 206/418] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:32.908742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.031408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.039463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49409c-80a0-4412-8a39-2b488d180d24 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.068253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616f7e-aa11-4b20-b6f3-c5f8e7ebad28 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.069032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616f7e-aa11-4b20-b6f3-c5f8e7ebad28 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.069032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61616f7e-aa11-4b20-b6f3-c5f8e7ebad28 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.071511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 207/419] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:33.075831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.076157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.076406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.089597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.090227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.090506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.090776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.091502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:33.091802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:33.093205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:33.094891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:33.096144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:33.111468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 213/420] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:33.117804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.172384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 208/421] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:33.191294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.288532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.309049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c337f-487f-4f21-acc9-18e6a3a15da9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.343018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.343901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.344357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.344777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.345360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba025-2222-4923-9834-6f585b437b0c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.356904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:33.362754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:33.363825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:33.365251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:33.372007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 209/422] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:33.373424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.420914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.421574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.421837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.422112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.471425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.481174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8628-d80a-4eb2-b17d-1454921ff51e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.516492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.517589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.518019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.518502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.519362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:33.519811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:33.522034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:33.534997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.535281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 210/423] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:33.537358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.544879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:33.546613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:33.643335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa86e-f664-49d6-8f07-e03148c9f7d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.679762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.680753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.681184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.681728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.722129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 214/424] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:12:33.728965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:33.791675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.818126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:33.834320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88a629-c5f5-4554-baeb-7a7a0a498825 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:33.840794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:33.842274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:33.868481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:33.869123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:33.869509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:33.869787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:33.870963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:33.872569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:33.874542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 215/425] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:33.882746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.013451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 211/426] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:34.018607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.118123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f90-00a9-463a-a940-30ee5f0f94ac None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.123110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.131154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b68b8-ee93-4ab3-a411-e48c897f54e4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.173526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.174171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.174451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.174723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] PATH_INFO: `/v3/projects/f00df1bf34b24986ab49f47aa7d3962c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.179597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] RBAC: Authorizing `identity:get_project(project_id=f00df1bf34b24986ab49f47aa7d3962c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.181065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:34.182900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 212/427] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/projects/f00df1bf34b24986ab49f47aa7d3962c => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:34.183078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.188987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.202264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.202918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.203197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.203471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.204755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.206207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:34.207833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 216/428] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:33 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:34.213755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.331492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02beb-f62c-445a-823d-f3f407fa61d1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.365021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a200acb-6eac-4dc4-9cdf-6ed607ad9d61 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.369563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.370182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.370466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.370740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.371540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:34.371847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.373441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d5755-b5f0-48cc-8ed1-722945627743 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:34.380967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 217/429] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:34.382282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6306-2d2b-414a-9836-bf6317ceccb2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.382571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6306-2d2b-414a-9836-bf6317ceccb2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.382849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6306-2d2b-414a-9836-bf6317ceccb2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.388557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 218/430] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:34.390009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.390299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.390576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.419645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:34.421179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:34.440688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.441427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.441712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.441986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.443476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.445105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:34.447275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 213/431] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:34.452888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.514104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.532267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.558574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 219/432] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:34.584210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.617417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c6cb6-ff87-4eff-ae5a-12bc4e11de22 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.664169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.665438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.666360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.667079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.668376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=f00df1bf34b24986ab49f47aa7d3962c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:34.669267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.671280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:34.679570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 214/433] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=f00df1bf34b24986ab49f47aa7d3962c&role.id=9a068445d06b48629659d5184c3e91bc => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:34.720921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f56b2-33a4-4750-8dec-e1ba54562960 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.760828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.761858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.762408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.762883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.776776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9bf189ac-beaa-472d-b3cd-8bdca056a5f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:34.782225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:34.783182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:12:34.783970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:34.786858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 220/434] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:34.794358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.807096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1e9e-14c0-49af-99cc-e5a4912e887a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.807406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1e9e-14c0-49af-99cc-e5a4912e887a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.807650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1e9e-14c0-49af-99cc-e5a4912e887a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.811101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 221/435] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:34.821949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.822257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.822511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.848674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:34.850238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:34.908346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.939992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:34.940269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 222/436] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:34.944924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af5fa-d5a4-4a34-acba-d5d2a91f52c1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:34.961047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:34.992074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:34.992707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:34.992989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:34.993270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:34.993989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:34.994305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:34.995904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.008818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 215/437] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:35.015095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.105199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df76fb-b32f-4efe-8119-ef3e039795d9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.140949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa5ea-d3ca-44b5-b5c6-3c2e14754d75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.162795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.162795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.163078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.163149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.177015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:35.182299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:35.182713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:35.183552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:35.184373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:35.184955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.184955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.184955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.185139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 223/438] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:35.190771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.225291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.242881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.281064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.282535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.335342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a7bd8-c372-4706-8577-fd9e69b989bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.374431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.375340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.375624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.375897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.376647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:35.376952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.378382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad8442-c3b5-4efd-94d1-25369f0943e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.386769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 224/439] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:35.387761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc33d5-110f-4054-be3c-62803e15e995 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.388110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc33d5-110f-4054-be3c-62803e15e995 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.388410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc33d5-110f-4054-be3c-62803e15e995 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.389938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 225/440] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:35.394675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.394977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.395299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.402599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 216/441] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:12:35.403727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.415592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:35.416840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:35.465702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.485750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 226/442] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:35.488964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.510858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cfa9a-4852-445d-955a-4ab5d9ea7ba6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.522142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.544886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.545499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.546044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.546315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.590971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbafdd-063b-4a07-a11f-d5eab6f44b56 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.611603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.613167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.624605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.625172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.625453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.625726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.626882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.628357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.630034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 227/443] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:35.630777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.674475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.715412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 217/444] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:35.716257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.725777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.730586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97cad0-70fa-43c2-9772-6d72b9f1c71e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.768571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.769221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.769529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.769805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.781464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:35.787050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:35.787888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:35.788791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 228/445] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:35.789443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.833431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dac8e-18e9-4b69-8e31-bc484d057c0a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.866022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.866653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.866924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.867191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.868338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.869748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.871302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 218/446] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:35.872056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.888384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b1813-6b17-46d1-91fe-f583bcbc9c5d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:35.898843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.924980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:35.925613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:35.925892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:35.926169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] PATH_INFO: `/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:35.927349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] RBAC: Authorizing `identity:get_project(project_id=6a4b2a6baf9044dbb5f9a64d22bbbcdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:35.928812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:35.930445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 229/447] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/projects/6a4b2a6baf9044dbb5f9a64d22bbbcdb => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:35.931184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:35.957222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:35.970295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25897969-9af0-449b-ba13-b6380e932e20 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.004079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.005177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.005450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.005713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.006412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:36.006709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.008102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.015909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 219/448] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:36.016690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.033266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf2995-e1b1-45a3-abcd-751c884f1a6e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.067941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.068572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.068857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.069138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.069927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=ed22ce3e92574f48b1aaa82fa1077beb, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:36.070236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.071661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.077468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 230/449] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:35 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=ed22ce3e92574f48b1aaa82fa1077beb => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:36.078159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.117636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:36.133010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e45e0-81dc-4631-b4f9-d79b669a8e73 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.172850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.173485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.173701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.174121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] PATH_INFO: `/v3/groups/476d317832c0447995cd8c7448193f6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.175422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] RBAC: Authorizing `identity:get_group(group_id=476d317832c0447995cd8c7448193f6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.177069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.177387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:36.178994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 220/450] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/groups/476d317832c0447995cd8c7448193f6e => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:36.184994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.189165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea2dbb-e6ac-4d33-b0d7-efaa1c619028 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.225423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.226049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.226760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.226760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.307407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299b4e-fe51-4b41-b366-20a81373968e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.313702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.315571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.346067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.346684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.347994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.347994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.348171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] RBAC: Adding query filter params (group.id=476d317832c0447995cd8c7448193f6e, role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:36.348428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.349806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.357540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 221/451] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/role_assignments?group.id=476d317832c0447995cd8c7448193f6e&scope.project.id=6a4b2a6baf9044dbb5f9a64d22bbbcdb&role.id=9a068445d06b48629659d5184c3e91bc => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:36.422713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:36.437162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 231/452] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:36.443549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.467896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:36.547313 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c94a-2007-4b45-8da5-df43f614ae0a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.583300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.583940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.584250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.584526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] PATH_INFO: `/v3/projects/8d75901b5a184df7b4b0c986ca5abc96` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.585687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] RBAC: Authorizing `identity:get_project(project_id=8d75901b5a184df7b4b0c986ca5abc96)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.587119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.589144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 222/453] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/projects/8d75901b5a184df7b4b0c986ca5abc96 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:36.594858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.695532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81f2f-8ef6-4349-a832-ea7f0214e562 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.729623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.730257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.730536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.730811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] PATH_INFO: `/v3/groups/c765bb79131c464e8758e806df48844c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.731931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] RBAC: Authorizing `identity:get_group(group_id=c765bb79131c464e8758e806df48844c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.733397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69718012-e71f-410f-ab74-7f7006411d4c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.734968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 232/454] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/groups/c765bb79131c464e8758e806df48844c => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:36.741261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:36.840067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:36.852456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2134e3-2e45-4cf7-b320-476374ec4d12 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:36.884599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:36.885201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:36.885484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:36.885759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:36.886544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] RBAC: Adding query filter params (group.id=c765bb79131c464e8758e806df48844c, role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=8d75901b5a184df7b4b0c986ca5abc96) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:36.886852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:36.888306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7712454-3c73-426e-8194-2b561dbc4357 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:36.893932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 223/455] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Aug 4 10:12:36 2022] GET /identity/v3/role_assignments?group.id=c765bb79131c464e8758e806df48844c&scope.project.id=8d75901b5a184df7b4b0c986ca5abc96&role.id=abf8a86280c54a0798604e6c88e690cf => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:39.443511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41720f-e62b-4320-85e2-8cc6cc63944c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.444048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41720f-e62b-4320-85e2-8cc6cc63944c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.444048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41720f-e62b-4320-85e2-8cc6cc63944c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.445698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 233/456] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:39.451035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.451866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.452370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.476091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:39.477661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:39.544269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ccc30-05e1-4207-9f83-e6ff53ca9c77 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.544617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ccc30-05e1-4207-9f83-e6ff53ca9c77 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.544823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ccc30-05e1-4207-9f83-e6ff53ca9c77 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.546709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 234/457] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:39.556050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339b9d-533d-4135-b817-8d474ac1a960 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.556465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339b9d-533d-4135-b817-8d474ac1a960 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.556847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339b9d-533d-4135-b817-8d474ac1a960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.572298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 224/458] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:39.589776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-da339b9d-533d-4135-b817-8d474ac1a960 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:39.591128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-da339b9d-533d-4135-b817-8d474ac1a960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:39.595771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:39.672290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:39.694407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 235/459] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:39.721872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dfd4e-e3e6-4731-9877-5ecda65068f7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.722158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dfd4e-e3e6-4731-9877-5ecda65068f7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.722517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dfd4e-e3e6-4731-9877-5ecda65068f7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.723844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 236/460] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:39.729797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.730076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.730347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.744956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:39.750568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:39.751771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:39.766125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf1596-6ed8-4234-b906-6ed6774bfc9c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:39.804431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:39.804992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:39.805365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:39.805644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:39.816910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:39.818530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:39.829052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 237/461] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:12:39.829939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:39.834845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:39.835665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:39.836540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:39.836540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:39.836540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:39.836540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:39.836540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 225/462] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:39.842772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:39.959878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdae828-9550-4684-88d5-a8ece1ba0cf5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:39.972537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbef-67f3-4cd6-b3d3-113e5bd0f959 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:39.979531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:39.999420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.000089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.000369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.000637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.007069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.007849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.008049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.008293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.008980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:40.009282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.010658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094429-d2b8-4764-b158-ef41b9d5a6b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.012031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.012611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.013492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:40.028774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:40.029643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:40.036853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:40.036853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:40.036853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.036853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:40.036853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 238/463] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:40.037130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 226/464] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:40.037199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.037262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c0b2-79a5-43ee-8606-55d0894d6f70 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.037262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c0b2-79a5-43ee-8606-55d0894d6f70 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.037262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c0b2-79a5-43ee-8606-55d0894d6f70 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.037262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 227/465] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:40.039090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c0b2-79a5-43ee-8606-55d0894d6f70 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.142608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c0b2-79a5-43ee-8606-55d0894d6f70 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.158467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894226f1-6bc3-4492-9ca0-d3c31db5cd0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.178249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.178862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.179149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.179522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.180241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:40.180544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.181920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.193612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 228/466] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:40.194883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.195730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.196209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.196651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.198758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.208739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.210370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5ff202e9-1bd4-4ad9-b84c-d3ea7188f676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:40.225529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:40.226343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:40.227166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:40.227166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:40.227166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.227166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:40.229052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 239/467] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:40.230338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.230614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.230884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.252682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:40.253957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:40.263494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.272502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.314376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfc89f-e9e6-4f3e-93a5-eae3a1d18f30 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.327012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 240/468] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:40.328030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.364722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.365700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.366170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.366632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.367498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.369228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.394513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 229/469] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:40 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:40.395651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.465807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659289d-f19a-486a-b66d-05dc63620665 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.509278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.510514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.511017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.511476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.512515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.514481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.538082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3fe54-ee94-4af3-b580-5026f674b51d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.541456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 241/470] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:40 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:40.542617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.550571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.561965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.598199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.598199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.598199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.599079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.599711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:40.600050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.601446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.615642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 230/471] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:40.621996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.681197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc183e9-9ee3-4c8e-b89d-bf1fec7aa62f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.720789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.721850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.722551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.723125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.734128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac68fa-c81a-4556-949d-4c820944ee76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.737499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.739987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:40.757214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:40.758224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:40.759323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:40.759323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:40.759323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:40.759323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:40.761118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 242/472] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:40.769209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:40.782998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.783853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.784294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.784704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.785552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.787724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470effa6-5092-40a1-8dd9-171a3623db18 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:40.815761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 231/473] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:40 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:12:40.848897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.885802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300acd18-2169-46fe-85c7-56b467903668 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.886152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300acd18-2169-46fe-85c7-56b467903668 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.886354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300acd18-2169-46fe-85c7-56b467903668 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.889049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 232/474] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:12:40.898267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.898559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.898810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.906707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce08dd7-934b-44e6-8179-0bc8b25d30e6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:40.924008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:40.925479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:40.969504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:40.972434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:40.972434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:40.972434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:40.972434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:40.972925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:40.972925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:40.975055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:41.005376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 243/475] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:41.018021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:41.056162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 233/476] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:41.082560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:41.129230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8cbae-3466-420d-a45b-11a8e99e78ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:41.153655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:41.180531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:41.181167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:41.181449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:41.181725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:41.182478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:41.183932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:41.216782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec7f1-a4cc-494d-a4b0-624b72529186 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:41.217237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 244/477] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:41 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:41.275176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:41.275825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:41.276136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:41.276415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:41.277156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:41.278551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:41.293270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:12:41.293270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 10:12:41.293270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'a033f4024e5b4642be2e1bcb94b8617c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 10:12:41.293270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 04 10:12:41.321085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:12:41.322029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:12:41.322912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:12:41.323733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:12:41.324622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:12:41.327236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'a033f4024e5b4642be2e1bcb94b8617c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:41.329156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 10:12:41.331019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:41.333009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 234/478] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Aug 4 10:12:41 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:41.333142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:41.499434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87884e52-4d04-4c96-a2ec-c0a38eb7cba9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:41.534107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:41.534765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:41.535044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:41.535315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:41.546717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b298028b-73ef-4d56-95fa-572cd7fd902a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:41.552900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:41.553874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:41.554759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 245/479] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:12:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:41.559906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:41.586277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:41.670309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ee1f8-ea70-42a4-bfbd-c9a0dba0e765 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:41.710134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:41.710673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:41.710957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:41.711232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:41.711980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:41.712295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:41.713674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd56cf3-9c7e-4ed5-954e-d1509b000b80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:41.722665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 235/480] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:43.114056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eccb3-ad01-4a24-9f40-55c6067189ed None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.114599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eccb3-ad01-4a24-9f40-55c6067189ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.114599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eccb3-ad01-4a24-9f40-55c6067189ed None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.116231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 246/481] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:43.121291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f8d0-5c92-4e36-bf97-d35494375b18 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.121682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f8d0-5c92-4e36-bf97-d35494375b18 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.121858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f8d0-5c92-4e36-bf97-d35494375b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.146135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-db73f8d0-5c92-4e36-bf97-d35494375b18 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:43.147326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db73f8d0-5c92-4e36-bf97-d35494375b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:43.224564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 236/482] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:43.263091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eccb3-ad01-4a24-9f40-55c6067189ed None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.320076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:43.348173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54608e3-3c65-481d-80fa-c1bd2d1180d2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.348681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54608e3-3c65-481d-80fa-c1bd2d1180d2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.348966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54608e3-3c65-481d-80fa-c1bd2d1180d2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.351295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 237/483] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:43.355146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.355438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.355717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.385837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:43.386940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:43.399419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eccb3-ad01-4a24-9f40-55c6067189ed None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:43.400331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:43.462439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:43.463085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.467772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.468083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.487032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:43.492009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:43.493955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:43.495827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 247/484] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:43.501866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.527791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 238/485] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:43.570587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.666111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:43.676936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a061515-a208-41b8-befd-e37b1cf0c729 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:43.733105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:43.733731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.734012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.734286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.735003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:43.735308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:43.736727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef5f30-9ece-4916-aa51-91c23743cd52 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:43.738084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1837f-96ee-43da-937a-a23c7809adf2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:43.750701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 248/486] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:43.752026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5ffb-6b9c-4816-afa9-9c1958aa91ed None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.752322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5ffb-6b9c-4816-afa9-9c1958aa91ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.752603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5ffb-6b9c-4816-afa9-9c1958aa91ed None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.753868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 249/487] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:43.758392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce5ffb-6b9c-4816-afa9-9c1958aa91ed None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.781566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:43.782594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.783184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.783659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.796519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7f9c2cb0-d7d0-425c-8a5f-6326a30f5c8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:43.803260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:43.804275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:43.805135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:43.807900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 239/488] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:43.810107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.810609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.811162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.818338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:43.832226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:43.833811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:43.876209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce5ffb-6b9c-4816-afa9-9c1958aa91ed None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:43.916063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 240/489] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:43.916945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:43.917543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:43.917828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:43.918097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:43.919426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.941902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:43.943398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:43.965424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:43.966364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:43.967196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:43.967196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:43.967196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:43.967196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:43.970316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 250/490] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:43.971374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:43.996210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.055024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376923b9-e684-4593-aa7c-4039081a5d4b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.090664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.097795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.099392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.099821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.100554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.101850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:44.102582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.104801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.110815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b703-9bfc-4f39-9c58-cae530290c7e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.116375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 241/491] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:44.117427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.153282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.154062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.154471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.154855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.167378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:44.173048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:44.174109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 251/492] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:44.175047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.242949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944cf1-db28-41e9-b0d7-0568336b9a67 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.256658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.280716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.281340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.281624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.281920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.282642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:44.282947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.284336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8baa7a-946c-4f5a-971a-40984910a583 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.291516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6a0ec-3145-427d-bf51-d89a7ed1d939 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 242/493] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2391-81f6-4743-ac67-1a44974a7206 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2391-81f6-4743-ac67-1a44974a7206 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2391-81f6-4743-ac67-1a44974a7206 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 243/494] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:44.300281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b2391-81f6-4743-ac67-1a44974a7206 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.333717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.334361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.334641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.334912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.345772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.347234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:44.361996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:44.362912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 252/495] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:44.364051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.369242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.416273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b2391-81f6-4743-ac67-1a44974a7206 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.451861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.452503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.452779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.453048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.453749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:44.454046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.455393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f4c5-8a4c-497a-8af7-32504e55b947 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.464070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 244/496] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:44.464526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a2f08-1678-4a09-938c-db4a7d775730 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.465295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27300cc-120d-499d-a364-ae0472361cca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.465605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27300cc-120d-499d-a364-ae0472361cca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.465898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27300cc-120d-499d-a364-ae0472361cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.483480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e27300cc-120d-499d-a364-ae0472361cca None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:44.484743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e27300cc-120d-499d-a364-ae0472361cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:44.487019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.499169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.499847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.500178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.500476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.508492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.509964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-060aed63-466a-42d8-aced-6a9e74344e2e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:44.522248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:44.523131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:44.524016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 253/497] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:44.524782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621782d-055e-4a36-8029-5147b05eada0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.524782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621782d-055e-4a36-8029-5147b05eada0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.525023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621782d-055e-4a36-8029-5147b05eada0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.526499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 254/498] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:44.527186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621782d-055e-4a36-8029-5147b05eada0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.553964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 245/499] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:44.554793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27300cc-120d-499d-a364-ae0472361cca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.603009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.633305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621782d-055e-4a36-8029-5147b05eada0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.657455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27300cc-120d-499d-a364-ae0472361cca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.666167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.666781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.667060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.667328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.668040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:44.668341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.669716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.681819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 255/500] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:44.682935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.695574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.696263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.696541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.696810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.706649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.708586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-618e4eeb-235e-475c-ac5c-8680b7d65501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:44.724087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:44.725091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:44.726365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:44.726365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:44.726365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:44.726365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:44.726365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 246/501] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:44.726673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.726954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.727221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.731533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.745014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:44.746308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:44.786835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a15798-352e-4d05-a01b-23bf1e16c5d0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.814799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 247/502] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:44.815658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.824922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.825845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.826281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.826707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.827706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:44.828051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:44.829577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:44.846278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.867409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 256/503] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:44.868263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:44.922017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bea8e8-1c2b-409d-a2f2-4b9a73d8169d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.955371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:44.956095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:44.956384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:44.956735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:44.965093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:44.968082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:44.968840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cabb-e9d9-4e52-bca4-2ea54d092d78 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:44.973974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:44.975047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 248/504] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:44.976041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.003745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.004926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.005263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.005501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.012464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.013903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:45.027036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:45.028110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:45.029211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 257/505] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:45.029875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.071401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.084112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90629-f70e-4227-bc64-303ac243d68a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.116761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.117382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.117662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.117938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.118635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:45.118938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.120332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.128520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a54949-02a8-4c51-8630-c8e2ecb3f93c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.131677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 249/506] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:45.132479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.161705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.162342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.162622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.162917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.173368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.174805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:45.188562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:45.189551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:45.190584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:45.190584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:45.190584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.190584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:45.190584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 258/507] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:45.190895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.205410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.241317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5fdd0-edec-4048-823c-9689429a5e49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.278382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.278983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.279265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.279534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.291801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15762452-3d51-49c7-bfeb-9e25bf47a341 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.297913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.326280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.326932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.327237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.327516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.328300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:45.328631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.330087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.343299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 259/508] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:45.343299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.348640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.350081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.418212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.444088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98089035-7d70-4a55-b1d1-d30ddf1e8f57 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.447594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 250/509] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:45.448442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.477756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.478372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.478677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.478954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.479765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:45.480138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.481682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.495323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.499755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 260/510] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:45.500586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.548484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04f70d-0139-4364-b192-2527e518f488 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.583208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.584013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.584398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.584776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.591654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.593192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.596638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc2109-d007-4930-9619-e2a82e967378 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:45.605825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:45.606832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:45.607745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 251/511] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:45.608374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.631200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.636362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.636972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.637248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.637515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.638191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:45.638487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.639858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.650914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 261/512] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:45.651711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.713254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5442-3b30-4d99-b81f-a570d41a9145 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.722021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.746620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.747189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.747473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.747751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.749688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdef8b-e888-41fe-9ff4-b95acae2a177 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.758885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.760391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:45.774302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:45.775266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:45.776257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:45.776257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:45.776257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.776257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:45.779983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 252/513] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:45.780799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.783986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.784632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.784914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.785186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.796475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.797926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:45.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:45.813031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:45.813890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:45.813890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:45.813890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:45.813890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:45.813890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 262/514] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:45.814157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.859207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.885558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a356070-f805-48ee-9158-3d1135ff5f3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.912961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f672-5b6e-42e0-bfa2-9594e3a43086 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:45.918207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.918818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.919101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.919376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.946802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:45.947411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:45.947689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:45.947818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:45.947984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:45.948776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:45.949078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.950501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:45.967210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 263/515] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:45.968018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:45.993515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:45.994963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.064281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3812-cbee-48d8-ae7b-0251a7893652 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.065748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.095007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 253/516] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:46.095900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.099523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.100261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.100540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.100806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.101470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.102877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.123011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 264/517] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:12:45 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:12:46.123813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.160350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.207966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825757-307b-4c61-b0fe-dfdc21feb59f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.242828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630c91b-e536-4b64-aa20-1d0401062c1e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.256939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.257927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.258383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.258851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.259563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:46.259868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.261343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.273416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 254/518] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:46.274236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.282618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.283241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.283523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.283795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.284522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:46.284833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.286248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.300293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 265/519] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:46.302053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.326445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.388663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e4a1e-6038-4a1e-bb38-29c01266f775 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.410375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.412357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95323f8-9745-4954-b94a-cb20ac741f6b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.425456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.426069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.426377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.426646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.446707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.447406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.447693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.447983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.458560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.460062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:46.476493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:46.477380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:46.478214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:46.478214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:46.478214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:46.478214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:46.478214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 266/520] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:46.478470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.505502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.506956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.549174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.582415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608582be-04e2-461f-b37e-da16f12fe567 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.615487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 255/521] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:46.617957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.623308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.623982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.624272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.624805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.633389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.634857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.638970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:46.646786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:46.647842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 267/522] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:46.648783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.744208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370d7f-0189-4658-8c56-41733a1af90b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.761932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06fa45-9595-4045-bada-c8bb0518066d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.781687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.782325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.782602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.782872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.790179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.791665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.792079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.799251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.799868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.800176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.800451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.801138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:46.801478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.802838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:46.804276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:46.805273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:46.806435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:46.810161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 256/523] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:46.810990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.823381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 268/524] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:46.823381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.901882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:46.923122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec2700-3b00-495a-9de8-a5e11a2c40dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.923876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc526d6-ea31-4e41-84ad-1392e9ae224a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:46.957490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.957734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:46.958289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.958475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:46.958715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.958885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:46.959113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.959281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:46.960160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:46.960530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.962015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.969415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:46.970851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:46.979706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 269/525] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:46.980818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:46.991200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:46.992301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 257/526] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:46.997908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.039945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.092334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ab805-7f29-40b2-9879-3e4d084eee74 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.105253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1366ac-5e5e-446f-bdeb-37a723e7ffad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.127739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.128625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.129005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.129544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.130861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:47.131271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.132915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.136290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.142002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.142713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.143066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.143410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.150369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.151948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.154460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 270/527] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:47.159282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:47.164411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:47.165221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:47.166121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 258/528] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:47.166586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.273554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2e1f2-f318-41f0-932c-3852dbca7ae7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.279733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3501d9-1605-4b39-a0e6-d4761814793e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.285474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.314293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.314910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.315193 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.315472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.317193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.317807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.318092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.318365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.319161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:47.319468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.320892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.332699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 259/529] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:47.333491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.389588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.396453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.398045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.438984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca9b39-a6ab-468f-bcdb-7327a5a88045 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.475717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.476471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.476754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.477030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.477821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=5f0130283f8f42bb94a894b5a1b55d3f) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:47.478306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.479774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.492323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 260/530] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/role_assignments?user.id=5f0130283f8f42bb94a894b5a1b55d3f&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:47.492323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.514789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 271/531] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:47.515742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.529859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.616799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f1bdd-60af-406a-92e3-106068ff649d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.640730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.653604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8270-247f-434b-846c-4d9a9b50552c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.668525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.669403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.669692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.669993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.671583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:47.672069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.673813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.693706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.694932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.695390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.695837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.700031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 261/532] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:47.701084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.709018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.710872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:47.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:47.728756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:47.729591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:47.732782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 272/533] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:47.733766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.811753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:47.812206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d786c-70d5-4396-9fe8-5c8add20f347 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.847886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.848880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.849339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.849782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.851170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd287-2657-4a69-ba48-56f09bff356f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:47.861062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.862784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:47.878863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:47.879782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:47.880654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:47.880654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:47.880654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:47.880654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:47.882911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 262/534] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:47.883885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.889338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:47.889963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:47.890294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:47.890579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:47.891445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=e60a8fe993054ee3a7563058956917d9) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:47.891754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:47.893276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:47.908351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 273/535] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/role_assignments?user.id=e60a8fe993054ee3a7563058956917d9&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:47.909493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:47.916470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.002469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775445fd-fe73-4b5b-9645-f7cff59fc726 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.050429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956d0c-23db-4929-839b-ffc9dd4b6dad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.088829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.089807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.090248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.090669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.091531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:48.092040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.093588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.108610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 274/536] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:48.112375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.120977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:48.129313 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.130581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.131122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.131408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.132257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:48.132561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.133997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.167660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 263/537] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:48.174872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:48.282222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa678bd-d187-49d6-b5e4-253cb0c9ab74 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.299940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.317538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9827003-c1cf-42b9-818e-4dfbe8ab0c05 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.323535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.324288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.324670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.325012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.332249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.334300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-44191259-62be-4aed-9420-c2c27057a044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:48.346029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:48.346944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:48.348165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 275/538] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:48.353936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bc620-765f-499b-b64f-77eb674ef6bc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.354280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bc620-765f-499b-b64f-77eb674ef6bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.354652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bc620-765f-499b-b64f-77eb674ef6bc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.359997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 276/539] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:48.359997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bc620-765f-499b-b64f-77eb674ef6bc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:48.364954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.365641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.365925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.366198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.367049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:48.367363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.368939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece93c8c-dd53-44c7-8a46-3a1482af9254 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.375606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 264/540] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:48.377070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.377345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.377626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.399915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:48.407343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:48.448759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.529844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 265/541] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:48.541145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bc620-765f-499b-b64f-77eb674ef6bc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.569798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:48.604045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.605105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.605597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.606217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.607620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:48.608276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.610333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.638846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 277/542] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:48.645521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:48.721151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.758099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a675c-1f80-48c4-b2b4-c8aca23d41d3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.792993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf483-228e-4626-9bd3-7d0dcea598f5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:48.807920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:48.835413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.837167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.838364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.839217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.841201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=d48c3743968a4f3da87a246e8f5f6e38) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:48.841990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:48.843932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:48.850134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 278/543] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/role_assignments?user.id=d48c3743968a4f3da87a246e8f5f6e38&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:48.851405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:48.853145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:48.853145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:48.853145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:48.865147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:48.870585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:48.871503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:48.872312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:48.875147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 266/544] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:48.881716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:49.070237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa71b7-1794-417b-ab82-26d81215cd04 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:49.115790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:49.116486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.116768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.117035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.117747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:49.118044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:49.119403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:49.131569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 279/545] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:49.137150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:49.299978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761818d-3817-492b-bc42-0cbd5dc118cc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:49.346608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:49.347224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.347556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.347842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.361972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:49.363442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:49.369586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:49.384222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:49.385065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:49.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:49.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:49.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:49.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:49.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 267/546] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:49.390964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:49.452942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:49.523101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17c8de-8899-4164-a22d-e5445d3af434 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:49.565737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:49.566417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.566707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.567634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.568369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:49.568675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:49.570076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:49.590338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 280/547] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:49.595408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:49.699294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc82ea-a6ae-4e2d-9f47-b13c22b89f54 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:49.738083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:49.738832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.739553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.739831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.747774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:49.749260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fbeff6d9-6550-4b33-9277-21353cfe1a71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:49.760577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:49.761520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:49.763425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 268/548] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:49.768749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:49.903162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:49.907643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287d17-5c40-480d-904d-dd459705893b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.907933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287d17-5c40-480d-904d-dd459705893b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.908234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287d17-5c40-480d-904d-dd459705893b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.909707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 269/549] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:49.914272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812028b-4628-41d6-89d3-f23eea31fdee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:49.915770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.916082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.916362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.949906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:49.951029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:49.974784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:49.975402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:49.975679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:49.976026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:49.976776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:49.977136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:49.978741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.010713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 281/550] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:12:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:50.015666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.046244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.077199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 270/551] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:50.113435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.170610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b99e2-cdfa-449d-b21e-f4d23b405514 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.235335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.235885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.236192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.236465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.284322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.299707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9cbaf-2139-423c-bd58-35ccfb521d82 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.360568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:50.362047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7ba5b-a1a5-48b7-816c-948be81c8d4e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.364058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.364759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.365044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.365317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.386259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cbfc778e-ca1c-4761-9389-519f60d15fe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:50.392185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:50.393082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:50.393964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 271/552] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:50.395820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fc2ea-4dce-4377-8949-79bda69d6849 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.395820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fc2ea-4dce-4377-8949-79bda69d6849 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.396101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fc2ea-4dce-4377-8949-79bda69d6849 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.397316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 272/553] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:50.401984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fc2ea-4dce-4377-8949-79bda69d6849 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.456179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.490091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 282/554] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:50.491529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.491823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.492138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.510694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:50.511888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:50.525224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fc2ea-4dce-4377-8949-79bda69d6849 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.564083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.565194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.565660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.567622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.568298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.569378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:50.570662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:50.572288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.582184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 273/555] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:50.583377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.589810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 283/556] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:50.589810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.703213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f87f4-e885-43a4-ace2-580bbf94e23f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.707661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6c115-9413-48d4-9128-cea784085886 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.711901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.743805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.744055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.744402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.744836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.745114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.745598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.746029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.746460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.756994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:50.758616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.759728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:50.761561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:50.774459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:50.775343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:50.776239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:50.776239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:50.776239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:50.776239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:50.776239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 284/557] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:50.776914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:50.780052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:50.781410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:50.782265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:50.782265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:50.782265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:50.782265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:50.784452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 274/558] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:50.785478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.824311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:50.887910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e595a-4457-49ed-b67f-d02538f689d7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.906020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca157-9f5d-49ac-b284-9eb52e3cdb0e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:50.923112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.923869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.924233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.924513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.935579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:50.940865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:50.941777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 285/559] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:50.943295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.949608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:50.950236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.950517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.950790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.951489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:50.951796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:50.953190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c070f9-4c0c-462b-944b-da0d40f679a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:50.965222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 275/560] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:50.966453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d374-f2c7-4152-9f10-4a968e3fddf3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:50.966836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d374-f2c7-4152-9f10-4a968e3fddf3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:50.967008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d374-f2c7-4152-9f10-4a968e3fddf3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:50.968525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 276/561] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:50.969224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d374-f2c7-4152-9f10-4a968e3fddf3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:50.989660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.063850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.069623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a6cbe-4be3-4586-807a-290a3ac106b1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.080808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d374-f2c7-4152-9f10-4a968e3fddf3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.117678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.117928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.118318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.118600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.118707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.118885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.119056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.119313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.119585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.119892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.120069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.120330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.121287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.121725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a6a8-db3b-4ee3-bb02-811e619268fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.134354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 277/562] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.134354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.142514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 286/563] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.143795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.144112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.144390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.163325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:51.164650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:51.220946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.234225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 287/564] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:51.235048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.252068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda4db-c69b-40d4-b5fb-ee6bf671e5f1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.286311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.286942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.287221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.287492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.295152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.296863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:51.307908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:51.309089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 278/565] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:51.310171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.330616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.344274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108931e-84bf-4bce-a57b-b8be6e5d7ec8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.378048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.378817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.379089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.379843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.389548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.390974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e30e167b-29e6-435e-be27-c4ee58fd2645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:51.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:51.411006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:51.411868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:51.411868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:51.411868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:51.411868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:51.411868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 288/566] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:51.412445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ab26-3c8d-4a4a-b538-deecb6745b48 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.412722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ab26-3c8d-4a4a-b538-deecb6745b48 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.412988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579ab26-3c8d-4a4a-b538-deecb6745b48 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.414196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 289/567] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:51.414871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579ab26-3c8d-4a4a-b538-deecb6745b48 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.421549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63095-56ae-4d39-aa13-6bd538098214 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.455923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.456580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.456851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.457121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.463710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.465174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.467551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:51.477032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:51.477919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:51.478816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 279/568] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:51.479284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.522631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579ab26-3c8d-4a4a-b538-deecb6745b48 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.556112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.556749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.557025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.557293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.564579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.568961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:51.574159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:51.575251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:51.576110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 290/569] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:51.576785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.582589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd945bc-300c-434a-a73c-fc6384828a24 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.615557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.616201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.616485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.616758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.617548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.617856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.619276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9c027-e16b-4d8d-8858-ce526c204da8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.654595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 280/570] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.656012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.656298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.656765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.673888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:51.674626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2001b-691f-450c-b80a-4d19c138c524 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.675048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:51.707598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.708242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.708522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.708795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.709479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.709785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.711139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.722296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 291/571] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.723088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.728026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.742175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 281/572] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:51.743070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.782718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.823078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026e16-f85b-465b-bfb6-0209d0cf4119 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.842387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea039156-de70-471a-8835-69eaac363cb3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.855901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.856532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.856808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.857074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.857807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.858107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.859514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.871915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 292/573] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.872712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.876442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:51.876587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:51.876943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:51.877217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:51.877945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:51.878245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:51.880329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:51.892620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 282/574] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:51.892620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:51.937866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:51.975386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885d7a3-7f7f-498e-9315-15c40d1fe2a4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.988722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ab0b-f47d-4b6d-9f59-ce713c0a1525 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:51.994770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.009171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.009822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.010100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.010366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.022136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.022400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.023024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.023324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.023571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.023661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.030822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.032255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:52.037146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.038135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:52.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:52.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:52.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 293/575] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.039324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:52.048610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.049659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:52.050619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 283/576] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.051430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.140560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6c45f-dedf-4060-a67a-ba2d22a54a55 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.147085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90981d9b-c4ac-49e1-8d07-ff342658112a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.157310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.173230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.173866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.174150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.174419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.182185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.182789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.183066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.183334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.184136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=6cc87ba3c4b9425d91a293211fbc7df3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:52.184443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.184733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:52.185850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:52.190010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.191047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:52.204424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 294/578] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.205125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 284/578] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/role_assignments?user.id=6cc87ba3c4b9425d91a293211fbc7df3&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:52.205125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.205261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.229648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.307872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd19-3a66-47b3-bb90-36ee07518942 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.326489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875478b-5876-4785-ab8a-86a5a1f3ef53 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.348133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.348790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.349071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.349356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.361802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.362417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.362696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.362973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.364319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:52.364656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.365615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.365926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.367071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.383247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 285/579] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:52.383247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:52.383484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.384847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.385693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:52.385693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:52.385693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.385693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:52.387900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 295/580] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.388733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.419191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.481495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.496319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ad86-0403-45dd-b58c-2ea20d86c042 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.501623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4373cf-c791-4b36-9c81-6eea03cd08d6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.535028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.536192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.536472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.536737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.537524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:52.537825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.538393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.539009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.539252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.539348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.539543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.540272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:52.540576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.541926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.551200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 296/581] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:52.552530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.558341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 286/582] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:52.559328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.658805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.665116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5cc7-37fa-4145-a496-f327f2c6f98e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.669956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4668-8968-49b0-a648-90df24632e3e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.702779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.703413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.703731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.704054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.704876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.705488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.705771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.706097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.706795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:52.707101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.708536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.711292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.712744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.725153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 287/583] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:52.725153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:52.725399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.726196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:52.727051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 297/584] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.727520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.742759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.849226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1d2b0-e66d-4483-8ffd-220016830b8d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.863848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8005-9435-4730-ba75-8f72f452bce9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:52.885018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.885661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.885942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.886213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.898160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.899058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:52.899650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.899769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:52.900161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:52.900427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:52.912059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:52.913502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:52.914918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.915732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:52.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:52.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:52.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 298/585] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.916897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:52.936707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:52.937594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:52.938515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:52.938515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:52.938515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:52.938515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:52.938515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 288/586] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:52.939014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:52.943684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:52.982497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.025533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c3b9b-76a7-47d4-b1a3-7e085723d077 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.053443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a48f-b431-4887-b0de-2a5f7ccdef5c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.061083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.061721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.061999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.062271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.071030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.072507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:53.084298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:53.085252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 299/587] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:53.086194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.093778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.094399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.094680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.095008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.095764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:53.096097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.097579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.115612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 289/588] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:53.115612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.179610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.197962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc9c-b915-4d7a-872b-a959f1c7a6bc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.219514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.220442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59964c-ead9-4c74-8f84-85a59876e776 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.242787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.243485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.243793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.244122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.244964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:53.245273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.246648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.261275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.261944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.262285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.262588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.263303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:53.263637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.269173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 300/589] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:53.269344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.269442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.282202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 290/590] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:53.284952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.385974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a908-f9d2-46a1-aebd-aa4f0124869d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.413589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e20c5-2c8c-4408-bed2-8714ed285c36 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.419230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.419840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.420172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.420444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.421487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:53.421788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.423209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.438360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.442652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 301/591] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:53.443514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.455579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.456212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.456579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.456907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/5f0130283f8f42bb94a894b5a1b55d3f/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.471774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=5f0130283f8f42bb94a894b5a1b55d3f, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.473870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.480723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.487833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 291/592] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:53 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/5f0130283f8f42bb94a894b5a1b55d3f/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:53.488731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.571448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39ae-cd07-471d-b6f6-b0512c5427ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.615225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.616544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.617024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.617491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.619302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5c75-f4cc-47bf-85b8-dfe0cedb0e2d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.625965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.627774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:53.642460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:53.643403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:53.644266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:53.649349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 302/593] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:53.649349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.672080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.672776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.673083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.673374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.681508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.683039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:53.696584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:53.697520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:53.698427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 292/594] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:53.698992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.740414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.763105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.770792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46712c-3387-45ce-b113-af67b55f72cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.807787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.808572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.808860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.809136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.810020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99943982-a69c-4057-a8fa-48089cff9f95 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.820247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.821886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:53.838597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:53.839509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:53.840406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:53.840406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:53.840406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:53.840406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:53.844063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 303/595] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:53.844968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.849675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:53.850302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:53.850585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:53.850862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:53.851628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:53.851935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:53.853400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:53.867802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 293/596] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:53.872772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:53.974723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:53.983283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf774b7-ad33-45fe-9ca0-303c881a669b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:53.992760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cb28d-fb14-4914-9e23-2192055e6d87 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.021518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.022449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.022830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.023205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.024069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:54.024475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.026039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.038061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.038694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.038979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.039257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.039698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 294/597] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:54.040628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.044901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:54.045321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.047172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.070166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 304/598] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Aug 4 10:12:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:54.071187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.080296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.169689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845094-b2c4-4f14-aa9b-c6ce3d860920 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.177403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fdc6-4791-4e1f-95ba-245cc4ed4aa2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.211236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.212128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.212524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.212897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/e60a8fe993054ee3a7563058956917d9/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.230481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=e60a8fe993054ee3a7563058956917d9, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.232045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.247231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.247866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.248255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.248588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.251554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 295/599] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:54 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/e60a8fe993054ee3a7563058956917d9/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:54.252578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.257807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.259432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.262810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a2f419c4-f09f-4d6a-999f-514e22929d7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:54.277447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:54.278409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:54.279276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:54.284014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 305/600] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:54.285383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f018f-0c56-48b0-8c35-70a28ec1feee None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.285715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f018f-0c56-48b0-8c35-70a28ec1feee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.285992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f018f-0c56-48b0-8c35-70a28ec1feee None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.288236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 306/601] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:54.289093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f018f-0c56-48b0-8c35-70a28ec1feee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.365440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.387458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68922-dfde-4342-b416-708c91d6fa8b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.406134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f018f-0c56-48b0-8c35-70a28ec1feee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.432609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.433512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.433939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.434214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.447467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.448130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.448368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.448470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.448671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.449432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:54.449737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.450813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.451166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a50ae0-06c6-44ac-a21f-4decf7dfc7f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.466357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 307/602] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:54.467736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.468060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.468338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:54.470728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:54.471575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:54.472606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:54.472606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:54.472606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:54.472606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:54.475186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 296/603] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:54.475186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.497356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:54.498918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:54.550949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.583904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 308/604] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:54.585167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.610081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21dd30-8d4c-4039-9497-1b96c37b7d76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.655848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.661158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.662808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.663238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.663656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.680944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=45bdd938403d46d6b7b026abd46c564d, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.682729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.704102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 297/605] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:54 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:54.705129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.709056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97910-e53f-4a13-8fb9-73778b33d817 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.762528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.763180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.763461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.763736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.764491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:54.764797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:54.766343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:54.786489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 309/606] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:54.788016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:54.919221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d2d75-0e86-4f34-8b89-814776f1401d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.926858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b021d-e70f-449a-a327-2271e1066ead None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:54.935846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.961609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:54.982470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.983116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.983391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.983661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:54.994072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:54.994719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:54.994997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:54.995260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.006352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.007783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.008679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:55.013622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:55.014495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:55.015315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 298/607] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:55.020823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:55.030819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:55.031736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:55.032732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:55.037407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 310/608] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:55.040258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.199981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8e26-e03d-43bc-929d-3b3bd994b56d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.216835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9969-a51b-452c-a7b6-adb491cf0377 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.259885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.260093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.260321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.260590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.261343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:55.261646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.263082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.278330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 311/609] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:55.284039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.291891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.292535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.292813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.293084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.293796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:55.294096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.295452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.315255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 299/610] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:55.321940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.356367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:55.383108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:55.485840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fa34-b2e7-4a1c-8ab3-328e9aba30af None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.523407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fd9d-3e21-4979-bd95-b5a073d61280 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.551601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.552626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.553110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.553579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/d48c3743968a4f3da87a246e8f5f6e38/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.581694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=d48c3743968a4f3da87a246e8f5f6e38, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.583217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68447b-a375-4ba8-851c-68ee3839d575 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.585061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.586024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.586534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.587060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.599414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 312/611] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:55 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/d48c3743968a4f3da87a246e8f5f6e38/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:55.601535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.602007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bb8-039a-4633-a656-08d43a8e6843 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.602517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bb8-039a-4633-a656-08d43a8e6843 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.602970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bb8-039a-4633-a656-08d43a8e6843 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.603407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.605677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 313/612] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:55.613927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.614252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.614479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:55.626473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:55.627413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:55.628337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:55.628337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:55.628337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:55.628337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:55.631397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 300/613] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:55.637973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.650892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:55.652692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:55.758972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 314/614] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:55.782582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.794014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:55.801101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:55.815468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e52ea-f5fa-46aa-b42c-83197c4c0f45 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.854691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.855321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.855602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.855872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.856603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:55.856910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:55.858487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:55.871558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 301/615] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:55.878592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:55.902848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e552512-90ce-45e7-8b47-b57fc7c94d65 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:55.976296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:55.977678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:55.978199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:55.978779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:55.993086 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:55.999676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:56.000670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:56.001490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:56.005832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 315/616] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:56.014347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.056160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30830d-83c1-4817-830a-cb7d1d877018 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.122800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.123811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.124318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.124812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.128310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.137592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.139233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.142565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:56.154300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:56.155460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:12:56.156638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:56.159273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 302/617] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:12:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:56.164721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.188754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797b0ee-0833-41d0-8211-4fdba265fd39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.254418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.255343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.255860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.256371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.257337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:56.257849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.259461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.268534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 316/618] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:56.280130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.371462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d894ced-7073-4723-ac7c-3ba058108cee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.439408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46b3d-b24e-4baf-8766-46e46e6c4437 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.476850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.477480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.477762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.478034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.478835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:56.479143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.480613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.483520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.484251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.484533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.484806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.499214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.508948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.515503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.526530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 303/619] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:56.531669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:56.532706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:56.533564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:56.533564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:56.533564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:56.533564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:56.533784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.533861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 317/620] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:12:56.540499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.661592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.713027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49311a-280e-40e6-ae70-40a65ecb2b11 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.713715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece541-de3f-4f0c-a407-a8aa256ed611 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.751718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.752770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.753320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.753811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.763432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.764089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.764373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.764823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.765617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:56.765890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.766007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.767290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.767815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.781527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 318/621] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:56.792825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:56.793719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:56.794569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:56.794569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:56.794569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:56.794569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:56.794569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 304/622] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:56.794853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.800890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.873576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.913341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daaa15-db60-43a3-889b-ac0f2f2cfd80 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.925414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:56.945257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67fa6b-6aa4-494c-9153-7478984e0d15 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:56.952922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.953537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.953811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.954078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.954762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:56.955084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:56.956505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b365b1-8cbb-465a-a2ef-34d2cc761c63 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:56.970005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 305/623] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:12:56.973927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0626-f4d4-4bb4-9c3a-a6c69214026a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.975421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0626-f4d4-4bb4-9c3a-a6c69214026a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.975697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0626-f4d4-4bb4-9c3a-a6c69214026a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.977967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 306/624] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:56.978606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4698-44e9-4449-91a1-9dbad8a7afed None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.979109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4698-44e9-4449-91a1-9dbad8a7afed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.979284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4698-44e9-4449-91a1-9dbad8a7afed None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:56.981097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 307/625] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:56.981773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b4698-44e9-4449-91a1-9dbad8a7afed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:56.995283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:56.995937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:56.996251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:56.996524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.005676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.007118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6f13f8d9-9022-4402-83e7-665c5340fd03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:57.021389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:57.022388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:57.023322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 319/626] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:57.023903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a56-19cf-4578-93eb-0612c08dad54 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.024017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a56-19cf-4578-93eb-0612c08dad54 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.024458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a56-19cf-4578-93eb-0612c08dad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.043605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-be660a56-19cf-4578-93eb-0612c08dad54 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:57.044824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be660a56-19cf-4578-93eb-0612c08dad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:57.095068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b4698-44e9-4449-91a1-9dbad8a7afed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.122127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 320/627] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:57.123675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.123985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.124271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.137300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.138302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.138858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.139382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.144112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:57.145923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:57.147626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.149110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:57.160529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:57.161410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:57.162282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 308/628] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:57.162851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.182315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.199219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.217980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 321/629] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:57.219030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.284943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea063b2-7361-496d-b4fb-6ebf08c735a8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.330003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.331009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.331522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.331803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.333191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:57.333645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.334927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.353408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea34ec-1abb-4340-93ee-499168e0725e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.358312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 309/630] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:57.358312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.401824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.402959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.402959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.403203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.413792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:57.418696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:57.419710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 322/631] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:57.420619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.454762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.468923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.482753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ca6c-0321-474c-8c30-c378878dce3f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.518806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.519414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.519689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.520048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.520783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:57.521083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.522495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.527883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077734-f719-48f9-96fc-4bc989571e02 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.534506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 310/632] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:57.539934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.563002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.563764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.564075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.564345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.575105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:57.580525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:57.581489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:57.582314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 323/633] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:57.583062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.650632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5f819-1a1a-4129-bbfa-d8400aca25d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.680301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.688555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.689170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.689451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.689722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.693073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06f920-819a-405d-8e0c-c39f421033eb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.708959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.730303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.730952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.731194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.731466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.732200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:57.732502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.733878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.742800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 324/634] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:57.743576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.766566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.768160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.850916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607cd7-880b-47d4-8bb1-bc9a6c46dd90 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:57.871215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 311/635] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:57.872358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.889878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:57.890554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:57.890837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:57.891115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/6cc87ba3c4b9425d91a293211fbc7df3/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:57.905521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=6cc87ba3c4b9425d91a293211fbc7df3, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:57.906963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:57.920265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 325/636] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:12:57 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/6cc87ba3c4b9425d91a293211fbc7df3/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:12:57.921309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:57.923745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.929887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:57.991226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e57da7-ee50-4c92-84f4-1fe14ab20c0d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.028240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.028969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.029331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.029667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.030581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:58.031088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.032580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.037686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008a6c7-bc84-4a18-a4b0-ec126e3c3bd8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.047565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 312/637] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:58.047565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.083730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.084417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.084697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.084974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.095180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.096703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-529bff7f-918a-46e2-a780-6974b6dd5824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:58.112299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.113202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.114671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:58.114671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:58.114671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.114671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:58.114671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 326/638] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.115521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb188f-a561-4d40-a872-bd70aeceeff2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.115521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb188f-a561-4d40-a872-bd70aeceeff2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.115783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb188f-a561-4d40-a872-bd70aeceeff2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.117170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 327/639] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:12:58.120742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb188f-a561-4d40-a872-bd70aeceeff2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.168925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e594f-018c-4d81-9cd3-4c010909f6a3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.178566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.181315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.209153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.209976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.210257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.210598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.221873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.224010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.227627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb188f-a561-4d40-a872-bd70aeceeff2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:58.239104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.240029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 313/640] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.245830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.269556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.270732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.271014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.271292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.282278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.283717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-934756a3-d526-4c98-9107-30795e281aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:58.299829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.301063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.301996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:58.301996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:58.301996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:58.301996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:58.301996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 328/641] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.302663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.302950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.303224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.321013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:12:58.322191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:12:58.353337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739f16-1add-411c-87e4-fcb72f279205 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.389468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.390014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.390298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.390575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.391262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:58.391595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.393002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.393125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 329/642] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:12:58.393903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.427732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 314/643] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:58.428586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.441363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.453799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.513283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d2c1e-194e-4cad-805f-0bfb949dcc5f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.531527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ad81-d101-4690-ba2b-723ed7f3c301 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.549215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.549845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.550125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.550402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.551113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:58.551420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.552814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.569986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 330/644] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:58.569986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.572436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.573058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.573337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.573612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.574300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:58.574605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.576012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.588571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 315/645] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:58.589374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.679304 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.681853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.690046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daeeb8-ba16-4584-9f37-a5eb8c902a13 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.702072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2eab8-699e-44e8-b385-4b52ed4f0bb5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.727723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.728414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.728696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.728981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.737667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.738520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.738835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:12:58.739466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.739881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:12:58.744212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.745316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 331/646] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.746253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.752034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.753641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:58.766094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.767049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 316/647] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.768569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.858593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862147a5-7c4f-4758-8587-05116ba066dd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.870508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b99b6e-3b33-4599-a648-2c9ff26442d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:58.894350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.895685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.895980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.896368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.903554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.903905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:58.904562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:58.904845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:58.905015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.905109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:58.912812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:58.914246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:12:58.918127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.919000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 332/648] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.920482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:12:58.931013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:58.931943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:12:58.932949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:12:58.937527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 317/649] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:58.937527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:58.940257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:58.945859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.045634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912881f4-a1c8-49be-8d2e-5b34cdb965fb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.052195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d777-898e-4ba3-b856-41ac33bfd5e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.085827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.086442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.086750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.087023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.087813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.087915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.088133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.088496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.088780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.089056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.089559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.089763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.090068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.091607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.106058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 333/650] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.106058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.107853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 318/651] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Aug 4 10:12:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.107853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.177350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.191144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.212859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca813-f67f-42bd-9192-042e845af70e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.222281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611625c-f1bd-4989-99d8-6e6f5bb4e660 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.253557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.254857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.255152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.255487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.256271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.256578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.257998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.265196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.265905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.266189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.266460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.267245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.267558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.269018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.275625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 334/652] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.275625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.296764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 319/653] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.296764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.388397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557155a1-5977-444c-b075-bd4c51215a4b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.394385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99308383-fe39-4af0-bcb5-020f7b6dea7c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.413435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.425033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.425674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.425962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.426784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.434124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.435018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.435786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.436323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.436648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.438338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.442091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:12:59.459908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:12:59.461032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 335/654] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:12:59.466806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.513514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.514960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.575405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd38d7-7198-4c97-ae2f-41010456a3ad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.609709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.610325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.610617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.610884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.611676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=5f0130283f8f42bb94a894b5a1b55d3f) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.612023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.613469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.620284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 336/655] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/role_assignments?user.id=5f0130283f8f42bb94a894b5a1b55d3f&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.621087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.621965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 320/656] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:12:59.622985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.647078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.700612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.755225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e58-8ce2-45ce-9721-e3f2286fba41 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.760949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b006-f0ac-4f66-aeca-9a037dfc1f51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.796553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.798535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.799179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.799677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.800605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:12:59.801146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.802705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.804783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.806281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.806564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.806840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.818577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 321/657] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:12:59.819741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:12:59.909143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.910402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5745d-065a-4694-abde-dd1b2a988aed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:12:59.920809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.945237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbf108-f415-4f44-a624-81b9a16d271f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:12:59.975203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:12:59.982286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:12:59.982926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:12:59.983204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:12:59.983474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:12:59.994925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:12:59.996492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:00.011708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:00.012614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:00.013478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:00.013478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:00.013478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:00.013478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:00.013478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 322/658] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:00.013756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.028981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 337/659] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:00.030483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cee7aa-6403-4f7a-a8a9-7537bb14707e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.030769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cee7aa-6403-4f7a-a8a9-7537bb14707e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.031041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cee7aa-6403-4f7a-a8a9-7537bb14707e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.032472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 338/660] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:00.032986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cee7aa-6403-4f7a-a8a9-7537bb14707e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.128576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbec671-9745-4b72-9d2e-3bbe96f8cf34 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.141401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cee7aa-6403-4f7a-a8a9-7537bb14707e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.157861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.163758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.164441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.164913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.165186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.174331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.175781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.178399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.179184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.179605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.180052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.180896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:00.181343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.182826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa4fd1-4761-4b17-8742-e9f1be824e94 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:00.187607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:00.189050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 323/661] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:00.190153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.203527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 339/662] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:00.205095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.205378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.205652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.224175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:00.225410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:00.234603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.300993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 340/663] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:00.303142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651ddd-4048-4d3c-ad2a-2521b27fe1f5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.304951 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.344803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.345606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.345986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.346354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.357745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.359292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:00.373924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:00.374977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:00.376085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:00.376085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:00.376085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:00.376085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:00.381811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 324/664] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:00.381811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.418310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84ea34-9d7d-4559-9f24-727ed1fd2eb6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.424251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.457099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.457807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.458085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.458353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.459141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:00.459442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.460900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.482515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 341/665] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:00.482515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.482515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.498569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caa591-1e2c-4457-8656-92e33d545020 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.532697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.533331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.533610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.533881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.540572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.542003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:00.554767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:00.555643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:00.556547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 325/666] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:00.557029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.588140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76acd-a323-4a16-abcc-29b374562d71 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.624819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.625454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.625729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.625998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.637716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:00.642611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:00.643581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:00.644422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 342/667] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:00.645142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.672634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce637-cc92-47d7-80e7-1685f89ff54e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.674317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.709182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.709825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.710117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.710385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.711056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:00.711161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.711358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.712845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.728761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 326/668] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:00.729635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.753114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d965b-0b60-4ed3-8fe6-0ac332eb9f12 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.789412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.790139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.790417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.790691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.836110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef770b0f-70cb-4f2a-bb4a-a5a9e7e0e44d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:00.863083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.865472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.875003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:00.875611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:00.875889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:00.876186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:00.876980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:00.877286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:00.878859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:00.895528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 327/669] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:00.895528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:00.920861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.934156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:00.974559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 343/670] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:00.975710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.006626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3bcc-dcb1-447e-b29a-06fea28a4697 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.040770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.041380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.041661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.041938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.042653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.042960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.044348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.053948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 328/671] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.054849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.086793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571000f0-653d-47bb-9c32-688f5db14fc4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.132372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.133293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.133778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.134206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.140633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.142337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.144010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:01.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:01.158288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:01.159358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:01.159918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301611c4-f06c-4b28-8874-a1d6250484eb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.161792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 344/672] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:01.162703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.191159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.196723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.197342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.197624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.197917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.198685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=e60a8fe993054ee3a7563058956917d9) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.198996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.200443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.206203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 329/673] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/role_assignments?user.id=e60a8fe993054ee3a7563058956917d9&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.206983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.313225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635013c-9d91-4fbf-850c-cf67c965f961 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.342880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf544f9b-5800-4def-b2e7-147b1d831cb1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.361991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.363353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.364320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.364810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.380319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.382134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.382738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.383375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.383714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.384028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.398372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.399851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:01.400921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:01.401768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:01.402630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:01.402630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:01.402630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:01.402630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:01.407434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 345/674] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:01.408315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.409029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:01.417457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:01.418438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 330/675] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:01.419299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.538156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9c437-cb59-436b-b5a7-c5e820542195 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.544088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9621-a8f2-4699-8fea-22cf90f01f7e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.556305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.574935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.575531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.575815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.576115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.576862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.577260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.578587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.591829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 346/676] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.592422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.592796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.593508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.593983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.594448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.595518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.596859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.599037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.615657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 331/677] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.616814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.684225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.699138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f514-de3c-4b38-9709-f5bcc154abf6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.728204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d6f7-17ce-48e5-99b9-d0b88fab8256 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.733958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.734577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.734855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.735125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.735816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.736146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.737521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.749670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 347/678] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.750502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.755158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.769946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.771389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.771905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.772426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.773422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:01.773940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.775558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.781970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 332/679] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:01.783063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.861114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e7cbb-ebda-4223-ac07-3e1abb987261 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.896927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc987932-7316-4924-9bef-c60e75501b7d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:01.909171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.910572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.910786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.911043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.920828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.923545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.926896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4c361797-e58a-4a03-8a4f-6e8870ab5f65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:01.940817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:01.942287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:01.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 348/680] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:01.944341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0c51f-0c94-49d2-a1ff-ede1f38d8d27 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.944550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0c51f-0c94-49d2-a1ff-ede1f38d8d27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.946327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0c51f-0c94-49d2-a1ff-ede1f38d8d27 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.946436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:01.946436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.946436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.946436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:01.946713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 349/681] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:01.948177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0c51f-0c94-49d2-a1ff-ede1f38d8d27 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:01.955349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:01.957138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3893181b-5cbc-41b7-954d-83779b9892c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:01.975221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:01.977287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:01.978241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:01.982151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 333/682] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:01.984073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:01.984073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:01.984073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.008293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:02.009552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:02.019997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.076107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0c51f-0c94-49d2-a1ff-ede1f38d8d27 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.097881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 334/683] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:02.098733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.122174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.122707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.122980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.123246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.124187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:02.124496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.127926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.144903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 350/684] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:02.146295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.245644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74308b-2390-4cad-a4dd-8b48549c1603 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.266571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.268433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc7456-2522-4205-9a16-fb1e9f27ef9c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.285194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.295345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.296441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.296821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.297437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.298774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:02.299205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.301902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.310982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.311867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.312312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.312729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.323541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 335/685] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:02.324655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.325219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.326789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:02.347641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:02.349318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:02.350215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:02.350215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:02.350215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:02.350215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:02.351987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 351/686] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:02.352911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.453318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaf699-ea75-44de-bfdf-166181c1e2d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.486057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932470-4d5e-4cbf-b6d3-5aa738016cb0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.493942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.494794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.495220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.495784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.496804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=d48c3743968a4f3da87a246e8f5f6e38) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:02.497292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.499016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.510062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 336/687] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/role_assignments?user.id=d48c3743968a4f3da87a246e8f5f6e38&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:02.511411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.528950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.529727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.530113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.530487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.573366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.588275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.643082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.647184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.720533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5b002-7545-4fbb-b938-84093cd08806 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.779638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.780290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.780568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.780839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.781526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:02.781828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.783198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.805082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 337/688] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:02.807334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.817562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 352/689] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:02.822717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:02.907440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.926890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:02.934418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db04a-32be-40a5-a971-7f684d528f43 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.953640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0489f91-8c83-42ef-9249-839fbe337b91 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:02.972845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.973884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.974485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.974963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:02.975896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:02.977616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:02.994553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:02.995149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:02.995506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:02.995821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.009258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.009258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.010024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 338/690] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:13:02 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:03.026038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:03.027025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:03.028281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:03.028281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:03.028281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:03.028281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:03.028281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 353/691] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:03.033249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:03.177975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ce5b-63b9-4e50-bb64-28d2185cd099 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:03.221297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:03.221919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.222201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.222475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.223846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:03.224131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.225489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.244212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 339/692] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:03.249632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:03.372152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13dc3-7ff7-4d86-8ed4-f91e095e2127 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:03.390499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:03.409034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:03.409729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.410005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.410391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.420708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.422474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-05301379-b441-4d94-8c70-4d227c900fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:03.439372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:03.440388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:03.441257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 354/693] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:03.447974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:03.464970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:03.505305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007159e-ee6e-454f-9f2d-0d3edcec8957 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.505305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007159e-ee6e-454f-9f2d-0d3edcec8957 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.505305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007159e-ee6e-454f-9f2d-0d3edcec8957 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.507085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 355/694] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:03.516039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.516527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.516947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.536271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:03.537732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:03.563736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce7a85-7445-4f06-885b-a9799b510d95 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:03.625567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:03.626536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.628118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.628898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.630378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:03.632144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.633900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 356/695] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:13:03.634093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.647643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 340/696] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:03.653461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:03.670528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:03.767780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23475-95c9-42a9-b121-4ceb3768812e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:03.778534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:03.806943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:03.807568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.807851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.808153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.808948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=6cc87ba3c4b9425d91a293211fbc7df3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:03.809363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.811095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.818306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 357/697] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/role_assignments?user.id=6cc87ba3c4b9425d91a293211fbc7df3&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:03.834048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:03.868783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7f32-b53a-4926-950f-4116102f1c50 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:03.919688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:03.920725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:03.921443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:03.921897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:03.922731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:03.924346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:03.939581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:03.940662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:03.941567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:03.944521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 341/698] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:03.950645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:04.063045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57c637-4bde-43ac-9d09-8776c051cb5a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:04.099407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:04.100357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.100792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.101218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.102048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:04.102501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:04.104118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aaf8f-aa4b-43bc-9bcf-8ea7e7f7f92a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:04.109437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 358/699] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:04.116544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:04.182564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174ee1-236c-437e-967a-428ebcae6266 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.183193 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174ee1-236c-437e-967a-428ebcae6266 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.183510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174ee1-236c-437e-967a-428ebcae6266 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.185215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 359/700] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:04.191421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.192323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.192615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.212104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:04.213291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:04.245414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef1c2e-4082-4c79-83c4-772d41abebdc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:04.254230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:04.303847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 360/701] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:13:04.329462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:04.427439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:04.428260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.428632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.429015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.429856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:04.430253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:04.431706 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:04.438779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 342/702] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:04.438779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:04.462388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411bbc-bbe4-4942-b206-bbe23a870662 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:04.503642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305145df-a489-497f-824f-6666577df1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:04.505153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305145df-a489-497f-824f-6666577df1ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.505587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305145df-a489-497f-824f-6666577df1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.506011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305145df-a489-497f-824f-6666577df1ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.518693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305145df-a489-497f-824f-6666577df1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-305145df-a489-497f-824f-6666577df1ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:04.524243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:04.525393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:04.526427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:04.528858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 361/703] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:04.534619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:04.689243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518426f-c1b4-4ea9-9d18-5f7b6b0e0508 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:04.733655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:04.734285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.734566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.734891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.735675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:04.736005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:04.737460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:04.751597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 343/704] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:04.757277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305145df-a489-497f-824f-6666577df1ba None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:04.797929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:04.901664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305145df-a489-497f-824f-6666577df1ba None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:04.947091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:04.948115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:04.948707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:04.949208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:04.967294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:04.969528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:04.993517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:04.994468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:04.995333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:04.995333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:04.995333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:04.995333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:04.997639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 362/705] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:05.003997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.097564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:05.149326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3d2c-3fd6-4ab5-a461-aeac3daf9787 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:05.187197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:05.187811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.188320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.188426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.189079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:05.189383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:05.190748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98a98-2611-4223-9f41-14d6284c39fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:05.203193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 344/706] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:05.211417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.304924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:05.328413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944b8a-32e4-42ac-8dfb-3a84b2d204b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:05.363298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:05.364944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.365430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.365919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.374201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:05.375925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f098b1d9-fd85-47eb-9465-4489121a6312 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:05.387986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:05.388957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:05.389892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:05.391318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8f63-6ab2-4eff-82a3-d9f8f1b8cf41 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.391602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8f63-6ab2-4eff-82a3-d9f8f1b8cf41 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.391871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8f63-6ab2-4eff-82a3-d9f8f1b8cf41 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.393148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 363/707] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:05.393439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 345/708] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:05.399978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.399978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.399978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.400803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec8f63-6ab2-4eff-82a3-d9f8f1b8cf41 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.428807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:05.430006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:05.580330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 364/709] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:05.603690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec8f63-6ab2-4eff-82a3-d9f8f1b8cf41 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:05.634367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.636004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:05.665694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:05.666328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.666607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.666881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.667678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:05.667996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:05.669424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:05.708075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 346/710] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:05.713487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.737524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:05.778098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d2fc-070d-4749-b50f-2c26a65541ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:05.825541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:05.826836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.827339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.828034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.829219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:05.831300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:05.835719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207015a-73ec-45fc-9add-4d612822dd36 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0e5dad66-1fa8-4916-b746-b998dd75d721 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:05.845804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:05.846773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:05.847611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:05.851298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 365/711] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:05.853366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00986e-d999-4e39-b5c0-e927455149cf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.853858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00986e-d999-4e39-b5c0-e927455149cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.854335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00986e-d999-4e39-b5c0-e927455149cf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.855813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 366/712] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:05.856725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00986e-d999-4e39-b5c0-e927455149cf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:05.886939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:05.887478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:05.887762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:05.888062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:05.943468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:05.966063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:05.967519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:05.973702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00986e-d999-4e39-b5c0-e927455149cf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.007273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.014024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.015336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.015830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.016712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.017886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:06.018396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.021153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3695244-6942-46dd-9ecc-a171f8e87a21 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.027066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 367/713] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:06.028476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.028977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.029490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.053071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:06.054576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:06.085810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 347/714] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:06.086685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.141736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 368/715] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:06.142902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.193263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.215697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69254c3-25b1-4231-afe2-cc5d5c4d5981 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.251127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.252068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.252526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.253682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.255080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:06.255571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.257152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.268286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 348/716] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:06.268852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d253bd7-cb9e-4fbf-bfe7-d41ef4834948 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.270355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.302009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.314056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.314356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.314751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.315098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.337253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.342109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:06.378637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:06.380195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:06.381092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:06.381092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:06.381092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:06.381092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:06.387993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 369/717] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:06.390215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.490080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287194-5efe-434a-8125-138266db5a43 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.538735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.539693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.540270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.540708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.541726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a31e6-a349-4bf1-b9e7-908a4e706526 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.557596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:06.562740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:06.563865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:06.569062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:06.571438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 349/718] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:06.577287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.581700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.595251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.596247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.596914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.598584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.599560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:06.600547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.602125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.631379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 370/719] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:06.636983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.721006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.732177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269505f-b8d2-4798-8e46-1e1706bb3be2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.795164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.796611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.797112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.797819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.799341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:06.800317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.803439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb548-bb64-4b55-a97e-bd8e3f7905eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.816752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0109a9-b189-40ce-894e-c6ac683bcea4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.824414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 350/720] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:06.826918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f27ef-c89c-47af-adcd-de079aa54d14 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.827761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f27ef-c89c-47af-adcd-de079aa54d14 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.828466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f27ef-c89c-47af-adcd-de079aa54d14 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.831329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 351/721] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:06.832450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f27ef-c89c-47af-adcd-de079aa54d14 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:06.856838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:06.857518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.857921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.858205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.872066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:06.874834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1a1d5ec4-85af-4ad8-a179-b9d057e53dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:06.888626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:06.889626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:06.890544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 371/722] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:06.891033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd13a7-1cf4-4807-b29e-1b0739eae158 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.891581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd13a7-1cf4-4807-b29e-1b0739eae158 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.891581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd13a7-1cf4-4807-b29e-1b0739eae158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.909536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-74cd13a7-1cf4-4807-b29e-1b0739eae158 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:06.911381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74cd13a7-1cf4-4807-b29e-1b0739eae158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:06.926707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:06.972501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f27ef-c89c-47af-adcd-de079aa54d14 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:06.985271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 372/723] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:06.987208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc1d5-6aff-44cb-af18-10e9916c7540 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:06.987495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc1d5-6aff-44cb-af18-10e9916c7540 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:06.987766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc1d5-6aff-44cb-af18-10e9916c7540 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:06.989904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 373/724] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:06.990600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dc1d5-6aff-44cb-af18-10e9916c7540 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.018372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.019271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.020016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.020301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.022535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.035125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.037169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-82ca60d7-a5e1-45b6-8603-7aa4f22cf9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:07.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:07.055084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:07.056639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:07.056639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:07.056639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:07.056639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:07.056639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 352/725] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:07.056993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.057290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.057511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.076910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:07.078161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:07.153222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 353/726] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:07.154467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.214943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.228791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dc1d5-6aff-44cb-af18-10e9916c7540 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.270636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f36133-b456-471c-95a0-758067d1586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.271256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f36133-b456-471c-95a0-758067d1586e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.271531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f36133-b456-471c-95a0-758067d1586e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.271802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f36133-b456-471c-95a0-758067d1586e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.272584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f36133-b456-471c-95a0-758067d1586e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:07.272886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f36133-b456-471c-95a0-758067d1586e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.274302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f36133-b456-471c-95a0-758067d1586e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.275244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a15751-0687-43e3-92b7-f1b7b14fc940 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.290825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 374/727] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:07.290825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f36133-b456-471c-95a0-758067d1586e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.321680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.322396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.322837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.323173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.334082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:07.338921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:07.340043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 354/728] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:07.345213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.384039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.397814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f36133-b456-471c-95a0-758067d1586e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.433034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.433697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.434078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.434446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.435228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:07.435635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.437096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.448686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 375/729] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:07.449438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.456115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551fd0-7b35-40a6-b27e-106153aba65f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.466921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.493236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.493945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.494342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.494721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.495551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.497052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.523990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 355/730] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:07 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:07.523990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.587683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2fc89-ec83-4c2c-896a-957846683409 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.637448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.638241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.638582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.638921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.639887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:07.640348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.642028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.654410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 376/731] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:07.655553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.668325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640372c5-f664-4901-a3bd-e7817f913113 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.676223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.722426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.722426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.722426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.722799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.723655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:07.724778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.726462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.735218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 356/732] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:07.745173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.788385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:07.823390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e5ab7-7e61-408e-81d7-5e1a1bd16bf7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.867216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65645-137b-4305-963e-6d818e7b0d4f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:07.873014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.873905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.874394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.874849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.886231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.888316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:07.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:07.905915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:07.907099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:07.908468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 377/733] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:07.910104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:07.910976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:07.911407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:07.911839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:07.913696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:07.925813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:07.928310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:07.957810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:07.958751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:07.959626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:07.959626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:07.959626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:07.959626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:07.962845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 357/734] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:07.968398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.053360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.060356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.082901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32623385-1225-42b0-957d-3768687a5b6b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.119282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.119907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.120222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.120498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.123979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:08.123979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.123979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.132492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a14b4b-059e-4e8c-af3a-fb94c9157faa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.140244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 358/735] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:08.141572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.220168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.221090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.221517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.221935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.223612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:08.224101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.225674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.259291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426590fa-b688-4b02-961c-11c662b89bb3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.267623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 378/736] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:08.273029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.303081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.305403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.305631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.305908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.320482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.322148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:08.338350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:08.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:08.340277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 359/737] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:08.352968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.356287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.429465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1214d6-f3e3-4ac8-8293-fa45bd1978b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.457783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.476273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.476902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.477189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.477459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.513987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229587b3-8104-4457-9840-455ef0256fd0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.548580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.549190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.549478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.549739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.550616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:08.550916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.553007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9fc32-5766-4a90-8223-ec792e6ccb3a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.570244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 360/738] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:08.571387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba26a3-ae6a-4d58-87dd-097412579a62 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.571670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba26a3-ae6a-4d58-87dd-097412579a62 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.571891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.572049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba26a3-ae6a-4d58-87dd-097412579a62 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.573332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 361/739] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:08.574148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a596233-36d6-4ab4-be6b-cca3479d8da9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.577922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba26a3-ae6a-4d58-87dd-097412579a62 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.640237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.683453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 379/740] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:08.684940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.685330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.685707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.690632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba26a3-ae6a-4d58-87dd-097412579a62 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.704949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:08.706282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:08.727837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.736727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396885d0-0286-4521-b0de-a5909917515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.739015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396885d0-0286-4521-b0de-a5909917515d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.739444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396885d0-0286-4521-b0de-a5909917515d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.740134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396885d0-0286-4521-b0de-a5909917515d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.777856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 380/741] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:13:08.784104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.836237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396885d0-0286-4521-b0de-a5909917515d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.837627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396885d0-0286-4521-b0de-a5909917515d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.908606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adef634-b079-4847-b09e-498465dfdaa0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:08.918275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:08.947465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:08.948581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:08.949056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:08.949560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:08.967760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:08.969416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:08.978931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 362/742] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:13:08.979835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396885d0-0286-4521-b0de-a5909917515d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:08.985152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:08.986223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:08.987107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:08.987107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:08.987107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:08.987107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:08.987683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 381/743] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:08.989190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.001586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.095972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b852bd-34f9-41ca-9143-fecd879d94c4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.105781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396885d0-0286-4521-b0de-a5909917515d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.131725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.132564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.132937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.133372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.144052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.144202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.144852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.145147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.145421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.146025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.146135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.147661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:09.161166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:09.163300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 382/744] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:09.165250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.165665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 363/745] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:08 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:09.171681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.213691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.303861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.377569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79732a5-b18a-43c8-b0aa-a2c24ba22377 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.404591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ee0e-229f-4355-8133-11f3edd902fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.417751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.418761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.419237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.419795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.420701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:09.421486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.423099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.436197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 383/746] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:09.442418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.444642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.445271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.445555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.445834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.446541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:09.446986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.448831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.461467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 364/747] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:09.473799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.596807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758260-5415-4620-ad6e-be3c31179946 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.600294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.633239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.644095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942fa8-00e4-4e13-a9fd-cee341702506 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.650378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.651394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.651867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.652627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.662987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.664957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:09.683760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:09.684905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:09.685752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 384/748] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:09.690846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.691496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.691822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.692127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.692859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.699587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.701063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:09.713482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:09.714386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:09.715252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 365/749] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:09.720326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:09.877657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc2c5-b70b-4e41-a6d9-dab37febc087 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.885336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83fb41-5527-49d0-8b9b-5e11e38e81fa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:09.938850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.939730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.941210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.942430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.945372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:09.945833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.948056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f113f-8b32-48ba-b936-95a5f092d2d9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.971364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:09.972505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.973043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.973508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.974895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:09.975425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:09.978509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fad21c-25ee-4d6f-8661-e7149ddd6f9e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:09.982141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:09.982720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 385/750] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:09.984432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675e31e-beee-4818-8fe6-0fc65ed2bdba None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:09.984896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675e31e-beee-4818-8fe6-0fc65ed2bdba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:09.985334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675e31e-beee-4818-8fe6-0fc65ed2bdba None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:09.991140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 386/751] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:09.992569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675e31e-beee-4818-8fe6-0fc65ed2bdba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.006505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 366/752] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:10.006505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.006505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.006505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.024314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:10.025608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:10.047023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.108959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675e31e-beee-4818-8fe6-0fc65ed2bdba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.120813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 367/753] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:10.121736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.146650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.147188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.147467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.147772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.148552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=55049003afb04ed4971d769ac2259efd) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:10.148860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.150295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28590d-ab25-4a72-8a2a-6a8ae5b30d8c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.156254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 387/754] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:09 2022] GET /identity/v3/role_assignments?user.id=55049003afb04ed4971d769ac2259efd&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:10.160231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58412e-bf3c-4f70-92a8-3f1580531a14 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.160231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58412e-bf3c-4f70-92a8-3f1580531a14 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.160231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58412e-bf3c-4f70-92a8-3f1580531a14 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.160231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 388/755] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:10.160231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58412e-bf3c-4f70-92a8-3f1580531a14 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.244211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d78f67-668a-4678-bd9c-eb8e80b537bd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.278040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.289703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.292042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.292515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.293133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.294784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=6cc87ba3c4b9425d91a293211fbc7df3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:10.296647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.299262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58412e-bf3c-4f70-92a8-3f1580531a14 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.299814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d5fff-5866-48c2-b807-65550067a823 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.304282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.312010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 368/756] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/role_assignments?user.id=6cc87ba3c4b9425d91a293211fbc7df3&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:10.313420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.313773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.314115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.341519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:10.343072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:10.345051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.346162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.346748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.347279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.348273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.349939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:10.361625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:10.362627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:10.364455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:10.367199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 389/757] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:10.373720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.434520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 369/758] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:10.462202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.507456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046963ad-4bfa-4308-9874-58d2cf643568 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.553731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.554892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.555423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.555917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.556925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:10.557444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.559168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.570458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 390/759] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:10.571717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.589440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274788c9-03d9-4c35-9ad2-da82317d6599 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.611763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.615478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.633564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.634665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.635155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.635650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.648028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:10.653730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:10.654651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:10.655480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:10.658234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 370/760] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:10.665182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.764982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd2ff7-b8dd-417a-a801-1453c687f45f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.813067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.814127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.814630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.815105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.816009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:10.816562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.818111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.825043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 391/761] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:10.830843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:10.872945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49717b7f-cf21-4749-a022-39711c2b8e62 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.933053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:10.935074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:10.935565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:10.936231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:10.936954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:10.937260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:10.938622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:10.941940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:10.951177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4693-a398-4db9-a162-0674cc2bae7c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:10.964850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 371/762] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:10.972043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.027625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.028196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.028478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.028752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.029512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] RBAC: Adding query filter params (interface=public, service_id=3218761a57eb440095b9201d947d01d2, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:11.029826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.031189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.050452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 392/763] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/endpoints?service_id=3218761a57eb440095b9201d947d01d2&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:11.055667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:11.056557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.150867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30fe2c-dcc6-4e7d-b92a-02df22e27371 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:11.178538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219d55-5717-4b1d-8a4c-dd2d53fb14a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:11.207768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.208716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.208998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.209271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.215239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.215985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.216447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.216823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.217592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.219025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.221768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.223649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.225032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 393/764] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:11.239545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:11.240561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:11.241441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:11.241441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:11.241441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:11.241441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:11.241441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 372/765] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:11.248564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.311451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:11.416352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933dfc0-d0a4-462b-b33d-daaca1981040 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:11.451737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.452388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.452668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.452942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.453635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:11.453940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.455310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.467671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 394/766] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:11.473819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.578143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:11.586238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedff1d-5242-4cb8-b0c5-8b1de30bd6f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:11.635556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.636240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.636524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.636797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.645747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.647187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f4df461f-f9c3-4aea-9602-407d0fe36f44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:11.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:11.666352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:11.667299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 373/767] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:11.673587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.723683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cbdb-43c9-45bc-a9ef-c524d393b57e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.724037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cbdb-43c9-45bc-a9ef-c524d393b57e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.724281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cbdb-43c9-45bc-a9ef-c524d393b57e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.730628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 374/768] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:11.732096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.732414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.732664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.763850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:11.765095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:11.827565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:11.844526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603832-cfac-4fd8-a60a-42bd30bd88a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:11.878525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:11.879169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:11.879446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:11.879715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:11.880534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:11.880833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:11.882247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:11.895210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 375/769] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:11.904216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 395/770] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:11.905738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:11.929494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.060312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.084713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ad799-62d9-4194-ad3e-2e444c280bcd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.105516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684569fe-60db-4624-9d2d-dfb5ce4c0645 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.146855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.147501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.147776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.148067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.148691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.152258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.153801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.154450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.154724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.154995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:12.166104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:12.167134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:12.168005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:12.172683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 396/771] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:12.180112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.182878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.184354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.195731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:12.216215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:12.218332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:12.220499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:12.220499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:12.220499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:12.220499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:12.220499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 376/772] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:12.225156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.320684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a5e8-eb68-4c6e-88af-dab3a0c0ee0e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.355842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.356423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.356699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.356970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.357644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:12.357942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.359299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.368372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 397/773] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:12.369726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.422004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e195e3-586c-4b40-9674-2a763146e53b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.440317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.471299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba180c8-cf9a-4010-bc05-375ba6d73f92 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.507356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.507988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.508662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.508934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.509608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:12.510089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.511441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.516902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 398/774] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:12.523229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.576690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044e465-ae87-4638-9063-a845de43766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.577333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e465-ae87-4638-9063-a845de43766b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.577617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e465-ae87-4638-9063-a845de43766b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.577897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e465-ae87-4638-9063-a845de43766b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.578800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044e465-ae87-4638-9063-a845de43766b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:12.579115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044e465-ae87-4638-9063-a845de43766b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.580533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044e465-ae87-4638-9063-a845de43766b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.606333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.610460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 377/775] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:12.617417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044e465-ae87-4638-9063-a845de43766b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.653756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d442a-fdc8-4453-a4c9-cf45b809050e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.702561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.703260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.703933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.704240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.705319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] RBAC: Adding query filter params (interface=public, service_id=55062bf8e3fb493d9038b95ec3cc8081, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:12.705631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.707095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4292-ecdc-4ab7-9e67-b548211fd093 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.715756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.716091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 399/776] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/endpoints?service_id=55062bf8e3fb493d9038b95ec3cc8081&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:12.724483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcaeae-3101-4de6-b857-87f3e4c4c5c1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.724681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcaeae-3101-4de6-b857-87f3e4c4c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.724921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcaeae-3101-4de6-b857-87f3e4c4c5c1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.731020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 400/777] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:12.731020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcaeae-3101-4de6-b857-87f3e4c4c5c1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.757683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044e465-ae87-4638-9063-a845de43766b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.792958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.793972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.794476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.794948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.802862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.804600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b12fdffc-b8e4-4003-82f7-2abdee31ef11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:12.816647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:12.817549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:12.818423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:12.820530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 378/778] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:12.821891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb438-2ba7-44f4-9e90-0c6e3438f999 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.822398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb438-2ba7-44f4-9e90-0c6e3438f999 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.822867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb438-2ba7-44f4-9e90-0c6e3438f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.841249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3cedb438-2ba7-44f4-9e90-0c6e3438f999 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:12.842298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcaeae-3101-4de6-b857-87f3e4c4c5c1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:12.842412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3cedb438-2ba7-44f4-9e90-0c6e3438f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:12.879994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:12.881075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.881075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.881075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.881676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:12.883039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:12.892441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 401/779] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:12.892441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:12.903464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:12.930157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 379/780] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:12.931773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a55b5-7933-4235-a5cf-157a2f137500 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.932284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a55b5-7933-4235-a5cf-157a2f137500 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.932698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a55b5-7933-4235-a5cf-157a2f137500 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.934441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 380/781] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:12.940444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:12.940990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:12.941651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:12.965942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:12.967213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:12.986736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.032287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39158-b851-460e-a12c-07d52c5fefd8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.047914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 381/782] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:13.049049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.098300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.099224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.099646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.100901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.102145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:13.103161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.105080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.120092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 402/783] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:13.120790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.182571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5ffb-dbf0-41e7-b463-18df7e510b2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.201330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.220887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.221563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.221843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.222115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.233282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:13.243908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e21db-00e1-4e3d-9bfd-150d14de243e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:13.248976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:13.249848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:13.251080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 382/784] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:13.251746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.274586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.288739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.289831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.290317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.290836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.302824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:13.308504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:13.309441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:13.310292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:13.317666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 403/785] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:13.318858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.364346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f5655-94ce-432e-972f-c33fc0ea1a44 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.418119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.418844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.419125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.419400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.421765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=45bdd938403d46d6b7b026abd46c564d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.423299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.433492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61edd-3e7a-4551-b68a-2ef918baf61b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.444733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 383/786] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:13 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:13:13.444733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.485357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.514249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.515308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.515839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.516372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.521431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:13.522059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.523711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.540624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 404/787] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:13.548388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.569691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2b6ac-22ef-4a22-9b18-91cb5f8a9c4c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.610038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.610861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.611328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.611700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.612559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:13.613022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.614503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.624862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 384/788] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:13.632815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.648377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.716476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da131eaf-3220-4571-9ffb-8998d169f530 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.755510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.756309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.756593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.757156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.759755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.767101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca77-8e88-46d3-8313-b7511dd735f2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.771665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.773250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0fd29705-57e7-4cb4-84d9-bcc61cb8a4cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:13.790362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:13.791493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:13.792722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:13.792722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:13.792722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:13.792722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:13.792722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 405/789] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:13.793100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382b44-439b-4bea-baed-1eb3d351958a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.793384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382b44-439b-4bea-baed-1eb3d351958a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.793665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382b44-439b-4bea-baed-1eb3d351958a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.800027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 406/790] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:13.802387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382b44-439b-4bea-baed-1eb3d351958a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.820701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.821991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.822739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.823230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.838905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.840838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b0cf5230-afac-42d0-940c-6fd0364a89b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:13.859757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:13.860830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:13.861689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:13.861689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:13.861689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:13.861689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:13.862616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 385/791] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:13.864195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.864693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.865181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.886364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:13.887563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:13.922484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382b44-439b-4bea-baed-1eb3d351958a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:13.944375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:13.959649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:13.960341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:13.960741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:13.961189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:13.961917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:13.962362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:13.963898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:13.973932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 386/792] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:13.974966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:13.983529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 407/793] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:13.989932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.064806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.087006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e06cd-f423-4bc1-837d-4797879ed70b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.110181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741501-1804-4921-b6a8-f30ff5b742f9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.124587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.125346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.125829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.126213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.127207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:14.127521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.129669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.146670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 387/794] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:14.146670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.153703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.154577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.155004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.155459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.163121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.164745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:14.178204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:14.179342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:14.180453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:14.182653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 408/795] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:14.183693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.204423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.263918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8fe0d-e71c-4e91-bd91-ff01becfd0f2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.296678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee8872-fd38-4966-9d29-1bc97ae9b32a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.300603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.301699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.301980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.302251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.302925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.304319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:14.313942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:14.315023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:14.315877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 388/796] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:14.316793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.321201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.341534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.342517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.343024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.343677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.352754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.354581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:14.367364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:14.368330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:14.369240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:14.377219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 409/797] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:14.378395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.423558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31bc0-3f80-4568-bc8f-8ee27d27ff0f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.460855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.461472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.461749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.462025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.462947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:14.463209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.464650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.465505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.485970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 389/798] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:14.487069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.492041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ad1e4-dd97-46c7-92c2-adcc30871626 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.529748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.530552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.530934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.531313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.532128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:14.532559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.534117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.548603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 410/799] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:14.549771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.558097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.600974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748a30-3e33-4ec8-b968-7b88b0ece8b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.638789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.640253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.640725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.641380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.642456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:14.642903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.645818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.668150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 390/800] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:14.669399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.680923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ab958-4e50-4a2a-9cc8-295f1c1035b6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.727514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.728253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.728535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.728857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.740569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.744173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.745676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:14.763390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:14.764432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:14.765453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:14.765453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:14.765453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:14.765453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:14.769283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 411/801] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:14.771074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.799077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88dfea-8e6c-4e60-8a44-2f44da252f49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.821036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:14.838625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.839500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.840083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.840535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.841426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:14.841883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.843389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.849877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 391/802] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:14.851011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.878849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f53b-e413-4e83-999c-075df4d7a572 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.914710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.915394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.915673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.915942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:14.927028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:14.928430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:14.942836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:14.943907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:14.944887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:14.944887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:14.944887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:14.944887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:14.944887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 412/803] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:14.945169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:14.960477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd5a78-b961-4b6d-affc-9b4f4710c84e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:14.997421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:14.998316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:14.998749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:14.999175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.000256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:15.000730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.002249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.015302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 392/804] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:15.016323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.016825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.051624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b4170-542d-4f83-9c50-bd609c5a255b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.065880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.086828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.087448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.087724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.088040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.088745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.090097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.119905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 413/805] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:15.120965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.129967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec157e2-a347-43a3-93dd-dee94de7e828 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.173501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.174422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.174855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.175280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.176208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:15.176676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.178275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.192365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 393/806] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:15.201521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.256451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.260483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392038-053a-4b79-a068-0a38fdf25dd5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.302706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.303314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.303564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.303805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.312060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.313508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7eb77b1a-3673-4498-a7c2-cca9aae248dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:15.329877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:15.330809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:15.333403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:15.336251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 414/807] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:15.337428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885168c-1d19-4198-9870-3590d31b37fa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.337715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885168c-1d19-4198-9870-3590d31b37fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.337983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885168c-1d19-4198-9870-3590d31b37fa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.339456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 415/808] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:15.342006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885168c-1d19-4198-9870-3590d31b37fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.349877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.353149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6a626-1c27-4d5e-95e9-a98c5f794cb9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.399386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.400493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.401066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.401353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.410853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.412622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-faacee4b-15a5-496f-beb8-f96d2a37deef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:15.430368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:15.431281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:15.432269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 394/809] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:15.433802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.433802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.433802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.457697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:15.459540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:15.463817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885168c-1d19-4198-9870-3590d31b37fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.501283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.501902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.502180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.502456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.503421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:15.503734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.505935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.524087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 416/810] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:15.524985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.541478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 395/811] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:15.542765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.548502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.657079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792927-8c14-492c-925d-b366d6f4fb32 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.658347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.670080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed59bc-56ec-49f9-ba20-0cb98ea9e8e3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.705961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.707273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.707734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.708216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.709704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:15.710468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.714360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.720604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.721275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.721550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.721821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/6cc87ba3c4b9425d91a293211fbc7df3/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.724325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=6cc87ba3c4b9425d91a293211fbc7df3, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.726200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fa7c2-ff4c-4726-8717-756a2d2a8486 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.727933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 417/812] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:15.732701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.744019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 396/813] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:15 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/6cc87ba3c4b9425d91a293211fbc7df3/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:13:15.744647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72e32b-9e5c-487c-94e1-0430048c634b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.744647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72e32b-9e5c-487c-94e1-0430048c634b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.744984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72e32b-9e5c-487c-94e1-0430048c634b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.746769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 397/814] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:15.747797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72e32b-9e5c-487c-94e1-0430048c634b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:15.835310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:15.859314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eaf73-1a62-4866-a3d5-54b2029ac2dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.905723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.906369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.906643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.906912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.907585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.908957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.917066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72e32b-9e5c-487c-94e1-0430048c634b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fe33f96a-8bd0-4913-b5f4-f9e4c0e52f57 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:15.919924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:15.920921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:13:15.922033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:15.922784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 418/815] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:15 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:15.924122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.924122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.924461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.945896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:15.947076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:15.958530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:15.959716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:15.960026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:15.960309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/55049003afb04ed4971d769ac2259efd/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:15.981726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=55049003afb04ed4971d769ac2259efd, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:15.983570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:15.995063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:16.001201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 398/816] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:15 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/55049003afb04ed4971d769ac2259efd/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:13:16.002169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.035411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 419/817] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:16.101037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.144290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8910e6-383f-49e5-a114-8fde3b684e38 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.189004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.190914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.191197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.191521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.195838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:16.196181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:16.198832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:16.209582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 399/818] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:16.217444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.262730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:16.300107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:16.305490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37544054-0743-4841-a4d3-8de6b95f426a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.344259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf70a14-7225-43c9-adc1-681d5f1d0ffe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.350719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.351410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.351697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.351991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.364108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:16.369367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:16.370323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:16.371383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 420/819] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:16.379990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.405610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.406410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.406798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.407307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.409635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:16.410051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:16.413404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:16.425421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 400/820] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:16.430257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.583714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f438-39b6-4e02-834a-13d936d0db40 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.584725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9233a12-a5a5-4f2d-b69b-73bf47bf1425 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.616333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:16.637175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.637806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.638131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.638411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:16.640522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:16.663982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.664617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.664894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.665164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.665880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:16.666177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:16.667578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:16.668998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:16.684013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 401/821] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:16 2022] POST /identity/v3/endpoints => generated 337 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:16.689916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 421/822] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:16.695802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:16.837795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0c2f-35e7-4f17-8798-3ead12038f82 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:16.882838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:16.883503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:16.883789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:16.884088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:16.900379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:16.901913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:16.918861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:16.919795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:16.920716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:16.920716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:16.920716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:16.920716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:16.920716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 402/823] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:16.927207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.058003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6b9f-96c8-4734-ad0a-a6e82d9f5ad7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.095526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.096112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.096392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.096660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.097375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:17.097673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.099103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.112694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 422/824] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:17.118150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.151671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.249261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59d990-5c6b-40a4-ad7e-28079e7d42d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.286857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.287497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.287815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.288122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.297272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.298770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0ed999ca-106c-450d-be53-2cb940def010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:17.315776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:17.317942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:17.320087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 403/825] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:17.322717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.389216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.433078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108c356-7b49-4ea3-96ae-c607849e98e7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.458952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f098-058a-44ce-ae0d-31b452a3a1e4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.458952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f098-058a-44ce-ae0d-31b452a3a1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.459769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f098-058a-44ce-ae0d-31b452a3a1e4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.461116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 404/826] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:17.469675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.469955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.470213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.474019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.474875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.475347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.475848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.476881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:17.477379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.478985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.493470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:17.494999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:17.500531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 423/827] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:17.509264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.579778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 405/828] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:17.602502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.625456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc5c3-a412-4ae1-a36c-6215a11a3cc1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.627198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.644387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.661979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.662633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.662907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.663186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.709773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82c9c-3065-43f9-af0a-f7fd04fa784b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.737127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.737127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.755256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.756324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.756814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.757289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.758143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.759721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:17.772154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:17.773162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:17.773992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:17.781928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 406/829] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:17.782396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.858197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 424/830] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:13:17.859127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:17.875853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.899332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:17.916057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45309911-449b-4831-87c8-ec02dac15e33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:17.951753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:17.952416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:17.952802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:17.953001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:17.953635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:17.953934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:17.955310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:17.964982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 407/831] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:17.966406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.001196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47191b-26cc-401f-aaaa-1ed8ef7ade78 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.045503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.046243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.046665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.046936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.058430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.059731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:18.075719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:18.076640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:18.077522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:18.077522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:18.077522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:18.077522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:18.081628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 425/832] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:18.082563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.088812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c7d77-8cd5-480e-b004-1c8685218f71 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.128499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.129439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.129720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.130142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.130825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:18.131127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.132534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2dde9-6027-426f-8560-55e5bc5c9f47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.136988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 408/833] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:18.138122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629ff9-4c1f-4692-bd40-37e096675f48 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.138400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629ff9-4c1f-4692-bd40-37e096675f48 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.138671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629ff9-4c1f-4692-bd40-37e096675f48 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.143991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 409/834] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:18.145634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.145926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.146205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.162387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.169854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:18.171106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:18.187417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.206424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a6a8-0050-484e-9c96-efdc59ff53a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.258041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.258939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.259366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.259817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.260694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:18.261181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.262816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.277025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 426/835] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:18.287052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 410/836] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:18.289053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.305714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.406703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754998fa-6df5-441b-94dd-ab6c403e9c06 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.424065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de313a96-262f-4bf6-9ee6-1766ac09ede1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.435033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.446794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.447454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.448034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.448317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.455987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.458131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.461825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.462482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.462769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.463047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fce28a57-470d-47ee-a42a-e48fe2f92ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:18.469366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:18.470208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:18.471079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 427/837] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:18.472252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6969a3-37d9-4ac1-b3da-e877f1e0c4bb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.472252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6969a3-37d9-4ac1-b3da-e877f1e0c4bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.472252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6969a3-37d9-4ac1-b3da-e877f1e0c4bb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.478573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 428/838] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:18.478573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6969a3-37d9-4ac1-b3da-e877f1e0c4bb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.481493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5c772177-9309-49c7-9027-acd20714f74f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:18.486475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:18.487421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 411/839] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:18.489681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.491453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.491453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.497294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.508163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:18.509409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:18.587979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 412/840] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:18.587979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.604733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6969a3-37d9-4ac1-b3da-e877f1e0c4bb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.640297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.641241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.641733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.642012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.642762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:18.643071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.644529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.664048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 429/841] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:18.664048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.701386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a01399-05c3-4a63-a6dc-841cf94e0588 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.709593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.738005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.738648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.738929 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.739201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.739940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:18.740273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.741636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.743249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.756079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 413/842] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:18.756779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.767801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d2152-0f25-40ce-8930-1447ced3808e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.803483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.804158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.804441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.804716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.815299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:18.821120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:18.822311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 430/843] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:18.823183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.865736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e9f2-f749-4452-a2e1-1b4622df20aa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.901546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.902163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.902447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.902721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:18.903507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:18.903817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:18.905260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:18.911609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 414/844] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:18.912425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:18.948198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42329607-8a15-44f7-804e-9f8e8e1ce842 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:18.958715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.993955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:18.994454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:18.994620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:18.994907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:18.995191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.007779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.009293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:19.028871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:19.029989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:19.031080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:19.031080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:19.031080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:19.031080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:19.032747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 431/845] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:19.045433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.053415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f387e-64d4-4836-bc5e-2251d3ae5d76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.092524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.093264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.093500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.093772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.094515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:19.094887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.096287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.109659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 415/846] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:19.110846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.165280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103e0d2-8a03-4413-8d51-0cd83ca1fdc5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.203386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.204036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.204326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.204599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.205292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:19.205596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.207088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.224152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 432/847] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:19.228783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.241693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad997f1f-e151-43d5-8650-189eae98554f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.258486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.279905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.295891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.297480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.297912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.298344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.314176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.317215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:19.335609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:19.336708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:19.337777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:19.337777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:19.337777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:19.337777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:19.339886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 416/848] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:19.345241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.354499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181cb0-7ed7-4a9b-bc31-86f4b0b4e1f3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.391177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.392046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.392426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.393493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.400778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.402412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:19.415871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:19.417473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:13:19.418597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:19.424818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 433/849] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:19.426897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.474476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2326041-63a0-4f2d-96dd-07286317c0c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.514255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97760-d017-4831-a537-722722f8571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.515238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97760-d017-4831-a537-722722f8571c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.515739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97760-d017-4831-a537-722722f8571c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.516358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97760-d017-4831-a537-722722f8571c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.517045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a97760-d017-4831-a537-722722f8571c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:19.517347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a97760-d017-4831-a537-722722f8571c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.518693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a97760-d017-4831-a537-722722f8571c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.525893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.532361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 417/850] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:19.538938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97760-d017-4831-a537-722722f8571c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.550241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c78c-e063-45a2-bd19-c2a699920fe4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.586082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.590201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.591099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.591606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.592121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.593916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:19.594474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.596297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.617455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 434/851] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:19.622582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.671731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97760-d017-4831-a537-722722f8571c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.708798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.710056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.710582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.711053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.719108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.720898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:19.733920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:19.734838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:19.735796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:19.738426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 418/852] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:19.744124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.747633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57791bd4-badb-4dd7-b158-a0278fd86e38 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.786027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.786706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.786990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.787265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.791150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.845591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:19.854594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a093f3-16be-4095-947b-917425cea13b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:19.863307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.865377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.890934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:19.891912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:19.892436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:19.892912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:19.893900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:19.894403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:19.896616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:19.915923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 419/853] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:19.924950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:19.976277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 435/854] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:19.982324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.023016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.043280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef4bc-a402-452f-a4fc-09b94b3707b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.079781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.080772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.081295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.081712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.093868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17f7a5-2986-493c-a282-3484606dfc4f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.105899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.134178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.135249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.135758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.136297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.148074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.149897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.155579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.157043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ef5bb-108a-423e-9d47-09e128e6a511 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:20.166927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:20.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:20.169114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:20.169114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:20.169114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:20.169114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:20.175931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 436/855] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:20.177053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.273158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 420/856] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:13:20.274467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9968-9ec3-4a05-adb9-9e805706cf5d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.274839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9968-9ec3-4a05-adb9-9e805706cf5d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.275178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9968-9ec3-4a05-adb9-9e805706cf5d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.276505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 421/857] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:20.281105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9968-9ec3-4a05-adb9-9e805706cf5d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.298362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.314441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81479a-54ad-4eea-8415-5e35bfffad93 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.338924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.351578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.352249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.352641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.352913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.353598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:20.353898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.355254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59c337-b9ad-4e31-a809-e57530469f6e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.371314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 437/858] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:20.371314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.371314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.371314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.387124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:20.388330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:20.393068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9968-9ec3-4a05-adb9-9e805706cf5d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.431045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.431949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.432250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.432522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.443108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.444556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.458167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 438/859] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:20.459885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:20.460772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 422/860] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:20.466404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.466738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.555236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.572399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcf54-c411-4545-a2d7-8f90dcf6b2f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.574884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.580474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df729-e555-4569-a6dc-377571d64cb8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.614579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.615379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.615754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.616061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.616158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.616758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:20.616875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.617039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.617132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.617310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.618612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.624683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.626590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.635595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 423/861] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:20.635595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:20.639135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:20.640127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:20.641268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:20.645553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 439/862] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:20.646441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.747784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632420c-c68c-43d7-986b-d3ceb991b6e2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.748569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050d3b-920b-4217-97bf-1dc563d5cde3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.783434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.783777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:20.784100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.784382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.784522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:20.784711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.784841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:20.785025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:20.785937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.787532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.791561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:20.793398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:20.796257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.804023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 424/863] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:20 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:20.804839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:20.807309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:20.808258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:20.809125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 440/864] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:20.825056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:20.858717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:20.952368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4f536-cf6f-4463-90b0-ff34e674f28a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:20.976019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5ace-a1ef-489b-b299-3c98e07c9c5b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.000502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.001122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.001400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.001673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.002420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.002721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.005395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.022342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 441/865] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:21.025034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.025732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.026011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.026316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.027075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.027539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.027841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:21.029287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.050268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 425/866] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:21.056828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:21.096482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:21.134894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71d25-cbe9-4a9e-859d-a6930fe28dd8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.141848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:21.163910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945990ec-5cb5-4ad1-aaad-e5a7ba6421ad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.168301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.168917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.169195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.169465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.170251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=55049003afb04ed4971d769ac2259efd) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.170559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.172264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae56996-71b9-4761-b3e5-8af814836a99 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.179029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 442/867] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:21 2022] GET /identity/v3/role_assignments?user.id=55049003afb04ed4971d769ac2259efd&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:21.180352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54ccb-2ea5-4ff0-9c19-34e620c4efc2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.180707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54ccb-2ea5-4ff0-9c19-34e620c4efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.181070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54ccb-2ea5-4ff0-9c19-34e620c4efc2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.182517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 443/868] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:21.188002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.188423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.188776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.209035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:21.210601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:21.213368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.215360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.215913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.216396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.217406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=6cc87ba3c4b9425d91a293211fbc7df3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.217934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.219567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.235778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 426/869] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:21 2022] GET /identity/v3/role_assignments?user.id=6cc87ba3c4b9425d91a293211fbc7df3&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:21.308717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 444/870] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:21.332099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:21.448828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02f2c-6787-4527-b9dc-639e0b3e2d5e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.484516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.485159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.485439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.485713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.486345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.487710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:21.497254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:21.498211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:21.499260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 427/871] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Aug 4 10:13:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:21.504506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:21.560909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:21.611733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d6863-98bc-4dd0-b6de-29c9250437b7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.648785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.649408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.649689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.649963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.650640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.650941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.652350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d69931-f36d-42a6-bf58-2a7dfcf2cceb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.657405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 445/872] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:21.663468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:21.675310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:21.773736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14baf9-bade-4ba8-b589-a748bdc9f8c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:21.809152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:21.810211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:21.810640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:21.811055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:21.811900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:21.812374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:21.813866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15371fa-d2c9-4bb8-a69a-7ce935e59269 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:21.819538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 428/873] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:22.908421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0ab6-5a12-4401-8acb-1c0413c2cb25 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:22.908946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0ab6-5a12-4401-8acb-1c0413c2cb25 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:22.908946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0ab6-5a12-4401-8acb-1c0413c2cb25 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:22.911134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 446/874] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:22.916614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:22.917065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:22.917166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:22.936913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:22.938117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:23.012652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 429/875] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:23.034950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae0ab6-5a12-4401-8acb-1c0413c2cb25 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.144692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae0ab6-5a12-4401-8acb-1c0413c2cb25 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.161512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.180225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.180872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.181152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.181428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.182116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.183472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ea2573d3-7825-4aee-8c05-eda6fbd27d11 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:23.193110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:23.194090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:23.195506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 447/876] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:23.200273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.202439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.303511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f2168-4901-4239-84ad-0f9411c6209a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.322229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac654ae-9b78-4101-b39b-665b68e12439 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.322547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac654ae-9b78-4101-b39b-665b68e12439 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.322794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac654ae-9b78-4101-b39b-665b68e12439 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.324512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 448/877] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:23.330012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb60a3-a28e-47d6-a4ce-42a397cc1c8a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.330329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb60a3-a28e-47d6-a4ce-42a397cc1c8a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.330574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb60a3-a28e-47d6-a4ce-42a397cc1c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.344392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.345010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.345287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.345559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.346239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:23.346535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.348419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.353388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 430/878] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:23.353874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aecb60a3-a28e-47d6-a4ce-42a397cc1c8a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:23.355163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aecb60a3-a28e-47d6-a4ce-42a397cc1c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:23.359135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.429162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.430499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 449/879] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:23.442282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6a3ac-6f88-4273-885d-cb7a9ab5fe65 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.442868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6a3ac-6f88-4273-885d-cb7a9ab5fe65 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.443350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6a3ac-6f88-4273-885d-cb7a9ab5fe65 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.448834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 450/880] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:23.450182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.450694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.451167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.470891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc00c28-a39c-4777-a096-c8526135bd19 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.471165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:23.472347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:23.510620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.511234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.511515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.511790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.512510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:23.512818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.514167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.519334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 431/881] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:23.520096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.548416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 451/882] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:23.549586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.592826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.641283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2248d-346c-4033-870b-6fe8652c9990 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.658233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d51ee-7f75-4031-8848-1b906f218481 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.672536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.679430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.680031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.680672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.680672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.691125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:23.694745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.695408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.695690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.696036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:23.696290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:23.697194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:23.703507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:23.704214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] RBAC: Adding query filter params (interface=public, service_id=28063f9da4cc4930ad90ec32e93988c8, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:23.704214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.704341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 432/883] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:23.704341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.705294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.708791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 452/884] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/endpoints?service_id=28063f9da4cc4930ad90ec32e93988c8&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:23.709597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.815478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e824-65d6-4668-8727-9fbd486468db None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.818911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.821564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72822b69-46d8-45f1-af0f-2143b3b6e4b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.856571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.857308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.857653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.857929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.858094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:23.858646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:23.858744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:23.858946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.859042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:23.859258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:23.859880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:23.860336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.861255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:23.868701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 453/885] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:23.870391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:23.871329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:23.877583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 433/886] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:23.878358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.878432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:23.930303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:23.974000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e821c-0ee3-41b5-978e-41266e54b5a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:23.975197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51a98-419d-4e4c-90e7-a0720b0cb989 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.007074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.007684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.007988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.008233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.008341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.008873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.008980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.009157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.009432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.010225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.015473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 454/887] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/services => generated 1011 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:13:24.016267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.020823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.022290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-75976a88-e7c1-4be3-95c9-a0ede13b1944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:24.039291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:24.040306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:24.041174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:24.041174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:24.041174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:24.041174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:24.041174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 434/888] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:24.041892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e14f9-6999-46a9-8ac2-7f552ed03e4e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.042181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e14f9-6999-46a9-8ac2-7f552ed03e4e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.042463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e14f9-6999-46a9-8ac2-7f552ed03e4e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.043822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 435/889] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:24.049081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e14f9-6999-46a9-8ac2-7f552ed03e4e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.052021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.128136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7a84-5343-4555-a279-79339cf9ccad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.165991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.166615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.166905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.167038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e14f9-6999-46a9-8ac2-7f552ed03e4e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.167176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.167845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:24.168233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.169604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0881-fd87-442c-9f5f-1addb506fb66 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.174678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.175438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 455/890] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:24.176563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.176853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.177131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.199007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:24.200157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:24.213915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.214545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.214827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.215106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.215803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:24.216139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.217508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.229434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 436/891] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:24.230069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.282641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 456/892] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:24.292147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.309410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.353830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885c13b-3260-4a19-8c8d-b80af5af4c98 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.391286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac337b2-3e19-47af-b498-e03f300f44ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.399069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.400115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.400407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.400677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.401354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:24.401969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.403001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.407808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 437/893] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:24.408587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.444189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.445516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.446261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.446809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.450084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.461203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.463694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:24.481866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:24.482904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:24.483824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:24.487180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 457/894] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:24.493525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.542328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae948874-b0dc-4590-875d-3335e2b0ff9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.582275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.583384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.583887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.584378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.585259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.586126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.588045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.608191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b60d3-3f31-4b6d-9a59-0e2eea573ec8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.608905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 438/895] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:24 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:24.655288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.656299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.656752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.657201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.658253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:24.658772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.660786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:24.687171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 458/896] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:24.693057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:24.806926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:24.817805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9b8d0-8d32-4968-b151-7aedbe374550 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:24.853872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:24.854493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:24.854772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:24.855045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:24.924688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:24.926672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.024578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:25.031443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 439/897] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:24 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:25.037759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:25.149030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921e7b-3fea-4be1-a285-7cec28f8f764 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:25.186943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:25.187597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:25.187878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:25.188192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:25.195888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:25.200463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:25.202178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:25.218181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:25.219201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:25.220092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:25.220092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:25.220092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:25.220092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:25.220092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 459/898] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:25.227440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:25.329782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faafc1-8424-4a31-9d6e-d9b651b3107a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:25.363850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:25.364493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:25.364775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:25.365051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:25.365740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:25.366047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:25.367406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.379548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 440/899] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:25.389012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:25.494701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb276da-db6d-4b4a-8682-40d60a73c3ba None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:25.529260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:25.529905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:25.530189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:25.530464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:25.537845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:25.539296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:25.551539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:25.552424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:25.553308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 460/900] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:25.560082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:25.611769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:25.665442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabba6b3-4d55-49ea-a41e-e364534e9346 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:25.704462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:25.705102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:25.705391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:25.705664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:25.706435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:25.706743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:25.708347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dca8-285a-45b0-956d-c45ac5565c51 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.722447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 441/901] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:25.728883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:25.786478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:25.843199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2a85d-b116-408a-bf22-9b6d5b9ba53d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:25.878807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:25.879458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:25.879739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:25.880043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:25.880853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] RBAC: Adding query filter params (role.id=9015a308ce9d4b9fa729fe5e1a931c21, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=55049003afb04ed4971d769ac2259efd) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:25.881163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:25.882643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c6d1f-1c0d-4b8a-b2c8-b8a002862da4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:25.889428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 461/902] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:25 2022] GET /identity/v3/role_assignments?user.id=55049003afb04ed4971d769ac2259efd&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=9015a308ce9d4b9fa729fe5e1a931c21 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:26.066816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1decd9-7a36-47e3-bf4d-c39abaf06eae None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.067303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1decd9-7a36-47e3-bf4d-c39abaf06eae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.067303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1decd9-7a36-47e3-bf4d-c39abaf06eae None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.068910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 442/903] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:26.074334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4467-03f8-4802-8603-815338756c04 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.074724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4467-03f8-4802-8603-815338756c04 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.074888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4467-03f8-4802-8603-815338756c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.093804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aabe4467-03f8-4802-8603-815338756c04 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:26.094995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aabe4467-03f8-4802-8603-815338756c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:26.171934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 462/904] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:26.190351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1decd9-7a36-47e3-bf4d-c39abaf06eae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:26.302941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1decd9-7a36-47e3-bf4d-c39abaf06eae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:26.317206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:26.339251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:26.339886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.340187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.340456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.341079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:26.342422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:26.351369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:26.352504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:26.353307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 443/905] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:26.358685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4467-03f8-4802-8603-815338756c04 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:26.407242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:26.468248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4467-03f8-4802-8603-815338756c04 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:26.503097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:26.505666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.506874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.507351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.508260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:26.508763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:26.510542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:26.516073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 463/906] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:26.528088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:26.633851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89904f-c30a-45af-9357-7a07fa007b64 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:26.669120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:26.669732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.670005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.670273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.670948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:26.671248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:26.672957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b8de5-a3d2-4950-a89e-269f5431743c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:26.678336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 444/907] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Aug 4 10:13:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:26.683768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:26.699456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fd60-7c1c-447a-89af-f860953a7079 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.699739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fd60-7c1c-447a-89af-f860953a7079 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.700049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fd60-7c1c-447a-89af-f860953a7079 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.701654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 445/908] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:26.706533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.706814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.707605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.726510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:26.727727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:26.759639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:26.791218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f644dd-1756-420a-9659-1a678980e9ec None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:26.802070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:26.808446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 446/909] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:26.825880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:26.827808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:26.828866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.829440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.829912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.830922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:26.833038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04112c-8969-4aa8-b362-4560a3cd6d28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:26.864179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 464/910] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:26.904439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7e190-d74d-4406-892d-41212bb46626 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.904741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7e190-d74d-4406-892d-41212bb46626 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.905011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7e190-d74d-4406-892d-41212bb46626 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.906505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 465/911] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:26.912913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.913538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.914047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.934159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:26.936047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:26.938633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249513d0-ab93-4462-abd9-d9f37fd43478 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:26.986727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:26.987648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:26.989034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:26.989034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:26.990016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:26.991404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.016119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 447/912] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:26 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:13:27.041767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 466/913] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:27.067298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:27.104426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:27.194781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d68f8c-b049-4542-8259-8d78eaef72ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:27.229938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e23251-c424-424a-9250-4d57084d684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:27.230719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e23251-c424-424a-9250-4d57084d684d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.231101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e23251-c424-424a-9250-4d57084d684d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.231473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e23251-c424-424a-9250-4d57084d684d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.232235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e23251-c424-424a-9250-4d57084d684d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:27.233690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e23251-c424-424a-9250-4d57084d684d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-77e23251-c424-424a-9250-4d57084d684d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:27.247135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:27.248326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:27.249330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:27.256012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 448/914] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Aug 4 10:13:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:27.259780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:27.352853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:27.362137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a0554-b4e4-47aa-97f9-9e4d6c2262e1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:27.398182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:27.398836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.399088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.399360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.400076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:27.400445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:27.401821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.407164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 467/915] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:27.412960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e23251-c424-424a-9250-4d57084d684d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:27.510301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:27.527162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e23251-c424-424a-9250-4d57084d684d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:27.562598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:27.563257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.563540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.563857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.564559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:27.564860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:27.566477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.572588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 449/916] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:27.578133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:27.686786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d139d-e92f-459d-84c6-d66995731d51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:27.722328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:27.722977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.723251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.723520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.724374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] RBAC: Adding query filter params (interface=public, service_id=30819e00e165413bb76d5ad47821d384, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:27.724678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:27.726035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61c74-d675-400e-bbc4-994f276242dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.731529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 468/917] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:27 2022] GET /identity/v3/endpoints?service_id=30819e00e165413bb76d5ad47821d384&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:27.736782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:27.842611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d4b35-c897-46b9-8a8d-24276856be34 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:27.878922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:27.879543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.879823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.880128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.880785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:27.882142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b50536-cc01-4a67-a779-0ed13e8995f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:27.888782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 450/918] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:27 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:13:27.991148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb28c2-df92-470d-a714-bead5c8b9d83 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.991495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb28c2-df92-470d-a714-bead5c8b9d83 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.991670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb28c2-df92-470d-a714-bead5c8b9d83 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:27.993342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 469/919] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:27.999061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:27.999335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:27.999611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.019033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:28.020709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:28.033113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:28.093930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 451/920] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:28.122178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb28c2-df92-470d-a714-bead5c8b9d83 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:28.144038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:28.234975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb28c2-df92-470d-a714-bead5c8b9d83 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:28.272198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:28.272827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:28.273095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:28.273361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.283985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:28.289602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:28.290636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:28.291534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 470/921] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:28.296908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:28.403138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2147a1-9ce7-44c5-af1e-00c23bd98452 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:28.440240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:28.441220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:28.441756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:28.442244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.443166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:28.443681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:28.445268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:28.454107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 452/922] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:28.460554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:28.547652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:28.570224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9148c4-a3c3-4126-8a32-bce5756f1769 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:28.605510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:28.606216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:28.606501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:28.606813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.617933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:28.619387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:28.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:28.635406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:28.636346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:28.636346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:28.636346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:28.636346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:28.636346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 471/923] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:28.641572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:28.657309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:28.746446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e63e1-015b-48cc-8f29-ca398b0a04bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:28.781050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:28.782052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:28.782542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:28.783019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.783926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:28.784473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:28.786572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:28.803992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 453/924] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:28.808791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:28.925038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b559b8-a34c-497d-ac63-211232c89b6e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:28.959407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:28.960273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:28.960731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:28.961481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:28.961959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:28.970185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:28.971973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-79db49c2-2f28-4027-a9b7-17ff9f6c8989 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:28.984681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:28.992341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:28.993278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 472/925] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:28.993882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.057675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf03e7-1978-46c5-af9b-2f69951a9be3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.057997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf03e7-1978-46c5-af9b-2f69951a9be3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.058229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf03e7-1978-46c5-af9b-2f69951a9be3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.059564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 473/926] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:29.064500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.064786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.065060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.065437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.085697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:29.087691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:29.105257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3ca31-7a16-41f7-ac3e-a710aa7a2fe7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.147686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.148335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.148776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.149059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.150171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:29.150481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.151996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.175913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 454/928] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:29.176126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 474/928] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:29.185419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.195171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.310549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d06f-6fd9-4af9-8be5-577964ec579f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.310793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340151ae-b646-4d8d-b156-9965a864c9f2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.331772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.337841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.347087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.348236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.348453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.348587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.348777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.349087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.349370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.349468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.349637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.350866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.353445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING keystone.server.flask.common [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 10:13:29.360275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.362122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.370794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.sql.core [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:13:29.370794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 10:13:29.370794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [parameters: {'id': '712e4a52193c471b9b29465afd7aa790', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 10:13:29.370794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9d32b2fc-b5ac-42fe-8343-0924b10d8cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:29.378193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:29.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:29.380081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:29.380081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:29.380081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:29.380081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:29.384408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 455/929] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:29.385498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf5e3-60e5-4e41-b7e6-f8222463e7ed None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.385784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf5e3-60e5-4e41-b7e6-f8222463e7ed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.386056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf5e3-60e5-4e41-b7e6-f8222463e7ed None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.391553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 456/930] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a4ebdff1-ef4d-471c-bc85-5232b3715150 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 712e4a52193c471b9b29465afd7aa790.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 712e4a52193c471b9b29465afd7aa790. Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:13:29.391707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 10:13:29.392686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application yield resource Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise self.value Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application session.commit() Aug 04 10:13:29.393779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:13:29.394671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:13:29.397300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:13:29.399189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [parameters: {'id': '712e4a52193c471b9b29465afd7aa790', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.401433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:29.403439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 712e4a52193c471b9b29465afd7aa790. Aug 04 10:13:29.404990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.406110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bf5e3-60e5-4e41-b7e6-f8222463e7ed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.406258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 475/931] 10.222.0.72 () {60 vars in 1140 bytes} [Thu Aug 4 10:13:29 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:29.406258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.406258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.406258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.420509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:29.422391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:29.493348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 476/932] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:29.494551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.512512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bf5e3-60e5-4e41-b7e6-f8222463e7ed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.550154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.551401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.551898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.552385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.553246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:29.553784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.556036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.578511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 457/933] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:29.578511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.605814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4f98c-8a84-45f7-943b-9c9c5134c0bb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.615388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.619984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.646287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.646930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.647210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.647481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.654398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.656060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:29.668314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:29.669209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.670081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 477/934] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:29.671116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.689625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87d366-5ff1-49a5-84d2-5b9166448bd8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.726069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.726817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.727093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.727357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.727992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.729452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:29.739476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:29.740657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:29.741550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 458/935] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:29.742985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.779686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedca8e-6116-46b0-9a8e-46234b2093f3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.813826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.814477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.814812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.815085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.822424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.823870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:29.835272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:29.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:29.837159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 478/936] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:29.837650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.851105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abd82-113c-4389-be63-bb134b72b2fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:29.852808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.868848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:29.894747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:29.895806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:29.896334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:29.896718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:29.897444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:29.897749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:29.899245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:29.927988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 459/937] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:29.927988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:29.966869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105101f2-13c6-4f1e-834c-6443a8ef7855 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.012944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.014100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.014380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.014913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.015627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.015933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.018342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.029005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 479/938] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:30.035549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.062146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8db35-32e8-46e3-b109-2b5d76347066 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.101488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.102258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.102475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.102743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.103496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.103799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.105865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1068d-8e9c-404d-aa9c-29e3a236a2bc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.119655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 460/939] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:30.120906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1be0-6deb-4eb3-8893-d30c86750c42 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.121320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1be0-6deb-4eb3-8893-d30c86750c42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.121951 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1be0-6deb-4eb3-8893-d30c86750c42 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.124072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 461/940] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:30.125701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1be0-6deb-4eb3-8893-d30c86750c42 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.139348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.150629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348779-ccfc-422d-a210-2c482929dee0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.152683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.190783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.191433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.191867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.192181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.192977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.193280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.194647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049137-4582-4810-8af9-fbeddccbe1ed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.200046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 480/941] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:30.201259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.201596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.201873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.219670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:30.220884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:30.249078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1be0-6deb-4eb3-8893-d30c86750c42 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.287743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.288366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.288650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.288930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/55049003afb04ed4971d769ac2259efd/roles/9015a308ce9d4b9fa729fe5e1a931c21` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.290446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 481/942] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:30.292102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.297048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=55049003afb04ed4971d769ac2259efd, role_id=9015a308ce9d4b9fa729fe5e1a931c21)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.298805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.317626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 462/943] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:30 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/55049003afb04ed4971d769ac2259efd/roles/9015a308ce9d4b9fa729fe5e1a931c21 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:13:30.318831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.405490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.429822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247028a1-6587-4eb8-bfe1-6e3dea2f64de None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.432604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.456870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334f75-78ef-47d4-b268-12ae7c690bdd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.478420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.479245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.479628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.479918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.480776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] RBAC: Adding query filter params (interface=public, service_id=ac15f81e7fc7442b8d85f2632a447422, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.481091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.482458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.490827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 482/944] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/endpoints?service_id=ac15f81e7fc7442b8d85f2632a447422&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:30.496916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.502598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.503255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.503539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.503844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.504538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.505967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:30.515834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:30.516851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:30.518075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 463/945] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:30.523657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.622931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bd557-f1ac-469e-b72b-9eee6c83b1de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.650220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fcbd5-ad9e-4dcc-ac90-280a5b8fc201 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.666367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.667274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.667559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.667834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.668672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.670451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.684074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 483/946] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:13:30.699092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.699736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.700065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.700349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.701075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.701387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.702871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.709585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 464/947] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:30.715452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.732711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.831673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511ff2-cef5-4f23-84b4-08c339c041ad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:30.868833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:30.869455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:30.869735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:30.870009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:30.870684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:30.870989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:30.872402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67be39-6278-4830-a586-5d5c3462e28a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:30.877788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 484/948] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:30.883724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:30.903636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:30.995407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c569-c9a7-4ff6-8e09-262ca3576e94 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:31.032581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:31.033253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:31.033544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:31.033821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:31.034552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:31.035949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45541-934d-4668-bdb7-b26ec6d82a1b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:31.058180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 465/949] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:32.045289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0247a7-4d9a-4d23-a1c7-134577f807af None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.045813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0247a7-4d9a-4d23-a1c7-134577f807af None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.045813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0247a7-4d9a-4d23-a1c7-134577f807af None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.047665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 485/950] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:32.053565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.053981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.054163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.072748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:32.073912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:32.134443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:32.149900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 466/951] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:32.174123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0247a7-4d9a-4d23-a1c7-134577f807af None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.271615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:32.300299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0247a7-4d9a-4d23-a1c7-134577f807af None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:32.335643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:32.336291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.336567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.336898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.337579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:32.339055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:32.354537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.sql.core [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:13:32.354537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 10:13:32.354537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [parameters: {'user_id': '0629b4ca9b3b424fb20d50ef98515c18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 10:13:32.354537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8b432a87-153e-42e6-9428-ae6abc4f48a4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() Aug 04 10:13:32.361522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:13:32.362467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application yield resource Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise self.value Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application session.commit() Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:13:32.363990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:13:32.364875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:13:32.365773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:13:32.366628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [parameters: {'user_id': '0629b4ca9b3b424fb20d50ef98515c18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:32.367456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 10:13:32.372116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.373765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 486/952] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Aug 4 10:13:32 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:32.374683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.451145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d0a40-15c5-45a1-88da-8c6c2215d4c0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.451448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d0a40-15c5-45a1-88da-8c6c2215d4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.451708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d0a40-15c5-45a1-88da-8c6c2215d4c0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.456212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 487/953] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:32.461425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.461716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.461989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.479301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3140ae-61b3-4997-a916-76030cde5438 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:32.481650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:32.483273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:32.524037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:32.524993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.525446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.525888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.536414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:32.538717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:32.554734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:32.555693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:13:32.556684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:32.560239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 467/954] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:32.561828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.562166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 488/955] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:32.585202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.627273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:32.669985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:32.690299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9aa5-79d4-4044-b411-c511b18a70bb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:32.706721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b610b-c979-4b62-91b2-1e7c0ad1bdcd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:32.734917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:32.736372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.736848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.737494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.738742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:32.739422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:32.741049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed2083-4c70-4757-a89f-de1f78ba1c8d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:32.743104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:32.743799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.744113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.744451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.755249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:32.760326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:32.761447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:32.762364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:32.766683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 489/956] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:32.767553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.787435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 468/957] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:32.810798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b375f0-c349-4048-aa67-7ab498eca3b9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.810988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b375f0-c349-4048-aa67-7ab498eca3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.817196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b375f0-c349-4048-aa67-7ab498eca3b9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.817196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 469/958] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:32.819049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.819595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.820099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.839887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:32.841478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:32.895495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736d8d7-a1da-4ab6-be91-ad3d21549c16 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:32.928992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 470/959] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:32.932785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:32.942602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:32.943238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:32.943524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:32.943801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:32.944574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:32.944885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:32.946502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:32.951378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:32.957107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 490/960] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:32.962727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.002446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.082566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4275f6-82ae-4b94-a308-68b7860058c4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.092821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be67a-578e-42d2-9ed1-46787efc35bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.121976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.122625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.122914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.123177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.123808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.125217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.131725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.132611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.133078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.133549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:33.134014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:33.134950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:33.135785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 471/961] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:33.140948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.149206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.151024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-19e3f4e5-a434-4c95-ba4d-c8436ad6e657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:33.166795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:33.167726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:33.168654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:33.168654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:33.168654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:33.168654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:33.168654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 491/962] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:33.172130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025c0e-3950-41b4-9484-f491f070f992 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.172411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025c0e-3950-41b4-9484-f491f070f992 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.172753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025c0e-3950-41b4-9484-f491f070f992 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.174023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 492/963] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:33.174705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e025c0e-3950-41b4-9484-f491f070f992 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.218269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.245456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21fc61-7f5b-4b76-a0c3-ec245f7c2f3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.249566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.279313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e025c0e-3950-41b4-9484-f491f070f992 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.280552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.281171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.281452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.281731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.282411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:33.282716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.284110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f3f1f-ea82-4293-a1ae-d0562ecebea0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.289259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 472/964] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:33.290385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.290673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.290943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.308705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:33.309914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:33.314294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.314907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.315185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.315456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.316174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:33.316475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.317824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.329270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 493/965] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:33.330048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.378150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 473/966] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:33.378894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.428712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589252-8a4c-4395-93ef-f60ea75d5dd4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.430225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.463364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.463997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.464281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.464554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.465218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:33.465520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.466869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.471528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 494/967] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:33.472330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.478591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.483119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426d81f-2f49-4ca1-becf-4c61f0a0a430 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.516274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.516907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.517185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.517458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.524881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.526307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:33.538081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:33.539151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 474/968] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:33.540111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.573192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2873d-458a-47b1-8005-5e93a7ad3916 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.607045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.607683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.607980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.608268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.608888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.610284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:33.619652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:33.620933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:33.622138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 495/969] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:33.622906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.640686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb0e1-30ce-465c-8cc4-6ad13855aa99 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.651237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.674877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.675488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.675795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.676108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.676803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.678151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.696768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 475/970] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:33.697555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.714605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.737624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2962fa1-5529-45c6-9aa4-4d8e686402e1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.789799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.790777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.791245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.791529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.792380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:33.792692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.794156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.813967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 496/971] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:33.815212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7771e-02f3-4fe5-b134-1bccd0ef08fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.820987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.858081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.858081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.858375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.858615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.859323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:33.859630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:33.861081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:33.867172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 476/972] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:33.872783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:33.911767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.928889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99522556-bf2b-48c8-8beb-3ddf3c4fed0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:33.963682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:33.972975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:33.973799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:33.974128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:33.974402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:33.987266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c47a4-46de-4a16-8edf-aae99e06ebae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.021481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.022104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.022386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.022660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.023335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:34.023641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.025038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.029589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 477/973] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:34.048289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.049819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.140353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:34.165942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 497/974] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:34.167819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:34.279068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a551b-811e-4d36-b1e8-672e514465f0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.313973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.314615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.314895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.315165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.325703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.327146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.328922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:34.343260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:34.344535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:34.345631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:34.345631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:34.345631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:34.345631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:34.346182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 478/975] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:34.353597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:34.461862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5e039-5a0e-4da6-8e9a-7059ad50fe34 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.495204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.495817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.496123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.496396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.497077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:34.497379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.498814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.511058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 498/976] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:34.516746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:34.616916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056c6a-1857-4435-a043-b9d11c146b6c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.650955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.651878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.652325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.652732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.662978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.665225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:34.683354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:34.684331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:34.685226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:34.688309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 479/977] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:34.690597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:34.737140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:34.791921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1766fd9-1b21-4aa6-9330-d2dd414ef724 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.825428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.826064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.826343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.826614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.827390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:34.827695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.829198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6496c49-5116-4204-850a-8fd10299bb31 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:34.847184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 499/978] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:34.849005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:34.905443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:34.952377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad7df-2031-4fe3-9e23-c2a44cdfb40d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:34.987399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:34.988129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:34.988565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:34.988948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:34.989820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] RBAC: Adding query filter params (role.id=9015a308ce9d4b9fa729fe5e1a931c21, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=55049003afb04ed4971d769ac2259efd) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:34.990222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:34.991876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33262c68-a458-4a50-b923-13da7b1cc16c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:35.004031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 480/979] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:34 2022] GET /identity/v3/role_assignments?user.id=55049003afb04ed4971d769ac2259efd&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=9015a308ce9d4b9fa729fe5e1a931c21 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:35.731540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abef70-2183-412b-8468-9083d2e02023 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:35.732088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abef70-2183-412b-8468-9083d2e02023 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:35.732278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abef70-2183-412b-8468-9083d2e02023 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:35.733478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 500/980] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:35.738656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:35.739067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:35.739250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:35.757233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:35.758436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:35.827055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 481/981] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:35.850175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abef70-2183-412b-8468-9083d2e02023 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:35.950713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abef70-2183-412b-8468-9083d2e02023 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:35.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:35.988073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:35.988683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:35.988963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:35.989234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:35.989862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:35.991225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ad774116-a1df-4e9d-8780-1a6b8ffe999d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:36.000185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:36.001160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:36.001982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:36.008027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 501/982] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:36.008247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:36.058354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:36.069311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5781dc-c677-4353-85bf-4f7f94f14944 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.069593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5781dc-c677-4353-85bf-4f7f94f14944 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.069867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5781dc-c677-4353-85bf-4f7f94f14944 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.071345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 502/983] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:36.075915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.076258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.076528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.094409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:36.095675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:36.116185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdbca0-61a8-464f-8f40-832ea9dd86b6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:36.154949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:36.155562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.155841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.156138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.156812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:36.157114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:36.158461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.164687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 482/984] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:36.170407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:36.176137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 503/985] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:36.198438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:36.283603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975a19-04ca-4c38-b44d-f4fd17b1361d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:36.298240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:36.301595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:36.305395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d47526-aa6d-44a9-bef0-de18c5c2b960 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:36.320448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:36.321149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.321433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.321707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.322376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:36.322678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:36.324042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.328987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 483/986] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:36.335676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:36.344292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:36.344911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.345188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.345457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.346530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:36.348309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.365075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 504/987] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:36 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:13:36.445340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd075e9-73c6-47cd-b293-82ebeea69102 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:36.487769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:36.488946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.489222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.489509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.490268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] RBAC: Adding query filter params (interface=admin, service_id=ac15f81e7fc7442b8d85f2632a447422, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:36.490572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:36.491930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4424dc-bbff-4480-b765-eb5900aaecf2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.505161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 484/988] 10.222.0.72 () {56 vars in 1249 bytes} [Thu Aug 4 10:13:36 2022] GET /identity/v3/endpoints?service_id=ac15f81e7fc7442b8d85f2632a447422&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:36.506375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:36.613507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285031c-6e26-46bd-805a-9e135deb2981 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:36.646820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:36.647441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:36.647723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:36.648023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:36.648675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:36.650040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a762a94-20e4-4b7f-9b9d-3516c699903e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:36.656410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 505/989] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:36 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:13:37.051583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77901-a908-4a6e-9e8b-99f7e7a60e86 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:37.052137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77901-a908-4a6e-9e8b-99f7e7a60e86 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:37.052214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77901-a908-4a6e-9e8b-99f7e7a60e86 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:37.053551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 485/990] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:37.058641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:37.059030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:37.059205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:37.078271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:37.079468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:37.097209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:37.155660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 506/991] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:37.173455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77901-a908-4a6e-9e8b-99f7e7a60e86 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:37.211062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:37.283357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77901-a908-4a6e-9e8b-99f7e7a60e86 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:37.318806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:37.319361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:37.319641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:37.319914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:37.320562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:37.321916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:37.330766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:37.331788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:37.332702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 486/992] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Aug 4 10:13:37 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:37.339436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:37.438228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a4b98-4f04-4e04-b684-aba63f782312 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:37.471635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:37.472282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:37.472588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:37.472861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:37.473535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:37.473839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:37.475228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e82bd-9412-43c6-9634-489e32434200 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:37.484166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 507/993] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:37.486081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:37.591445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:37.594834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93717bea-2f40-4163-8ace-928b9553ca63 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:37.629580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:37.630203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:37.630473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:37.630736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:37.631407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:37.631709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:37.633093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eff6d8-3420-4502-92de-83f2ad9a4a3a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:37.644216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 487/994] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:38.400588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29079ba-6bc8-40b3-8895-f30b96b2d4cd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.401138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29079ba-6bc8-40b3-8895-f30b96b2d4cd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.401138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29079ba-6bc8-40b3-8895-f30b96b2d4cd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.403998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 508/995] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:38.407537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.407932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.408127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.427506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:38.428717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:38.495442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 488/996] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:38.517108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29079ba-6bc8-40b3-8895-f30b96b2d4cd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:38.536110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:38.621997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29079ba-6bc8-40b3-8895-f30b96b2d4cd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:38.655158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:38.655799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.656160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.656436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.657055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:38.658408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e07f6a6b-ad1b-46f8-8de5-22bb958de210 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:38.667897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:38.668891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.674925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 509/997] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:38.675739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:38.684760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc80d-88e0-4349-8435-be86641937ab None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.685044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc80d-88e0-4349-8435-be86641937ab None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.685316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc80d-88e0-4349-8435-be86641937ab None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.686774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 510/998] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:38.691687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.692286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.692286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.713651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:38.714862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:38.794333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:38.797838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 511/999] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:38.804264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aee90-56c0-40fc-aec0-70c3cecc54be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:38.819601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:38.831286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:38.839463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:38.840112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.840390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.840663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.841332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:38.841632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:38.842982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:38.848628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 489/1000] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:38.854070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:38.925915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620ae8-2ba0-43b4-a71a-8a890f504bb6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:38.954563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ddc3c-f95b-49ab-bae8-c4c61e9f8c34 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:38.959530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:38.960191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.960469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.960739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.961353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:38.962699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:38.971515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:38.973225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:13:38.974257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:38.978275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 512/1001] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Aug 4 10:13:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:38.979098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:38.996559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:38.997426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:38.997919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:38.998205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:38.998880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:38.999189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.000624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.010628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 490/1002] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:39.011837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:39.036581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:39.067021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:39.096883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b5b3-8a04-43c4-afec-3e41e0f9f608 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.132489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.133410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.133763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1bc52-f886-460f-85db-0921b0999f60 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.134108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.134566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.135523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:39.136051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.137434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.147574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 513/1003] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:39.148630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:39.175420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.176075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.176362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.176638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.177413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] RBAC: Adding query filter params (interface=public, service_id=8b3dc8321d4240e78fae5ffeecb0681e, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:39.177724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.179096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.189057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 491/1004] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:39 2022] GET /identity/v3/endpoints?service_id=8b3dc8321d4240e78fae5ffeecb0681e&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:39.189955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:39.263023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4e16-949b-41e7-bcce-c8a106b2dd7a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.279940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:39.296257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c3f51-3edb-4ffa-b3ae-6808b1941b60 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.300140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.300793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.301074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.301350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.302021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:39.302327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.303684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.313619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 514/1005] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Aug 4 10:13:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:39.314701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:39.318054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:39.336319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.337334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.337625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.337904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.338568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.340495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e0d0e-381e-41a4-9fef-b55681c29f4c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.350297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 492/1006] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:39 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:13:39.428221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d1bc-fdad-4b72-a2f4-62d356b65eb7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.469765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.470394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.470673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.470936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.471675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.473172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfa1d-da47-41e9-a4cd-426d86a6dee1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.495986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 515/1007] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:39.658366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2abd1-b92e-47ab-afa1-8a413ad458f3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.659240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2abd1-b92e-47ab-afa1-8a413ad458f3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.659530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2abd1-b92e-47ab-afa1-8a413ad458f3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.661590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 493/1008] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:39.667073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfb7b-f6c1-4c24-a0c5-f6e57b43f873 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.667428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfb7b-f6c1-4c24-a0c5-f6e57b43f873 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.667770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfb7b-f6c1-4c24-a0c5-f6e57b43f873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.686445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f50dfb7b-f6c1-4c24-a0c5-f6e57b43f873 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:39.687623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f50dfb7b-f6c1-4c24-a0c5-f6e57b43f873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:39.746084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:39.768013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 516/1009] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:39.787262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2abd1-b92e-47ab-afa1-8a413ad458f3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:39.890673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2abd1-b92e-47ab-afa1-8a413ad458f3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:39.925384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:39.925926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:39.926205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:39.926479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:39.927178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:39.928570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3718c5-fa83-49e8-8ed0-2128d9755731 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:39.949715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 494/1010] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:39 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:41.378489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5eba-768f-4d1d-8732-62d72e63607a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.379618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5eba-768f-4d1d-8732-62d72e63607a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.379618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5eba-768f-4d1d-8732-62d72e63607a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.381091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 517/1011] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:41.386057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1e52-9508-468f-b55a-4d0da2fcd408 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.386478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1e52-9508-468f-b55a-4d0da2fcd408 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.386642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1e52-9508-468f-b55a-4d0da2fcd408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.405258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-91ca1e52-9508-468f-b55a-4d0da2fcd408 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:41.406433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91ca1e52-9508-468f-b55a-4d0da2fcd408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:41.409277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:41.480410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 495/1012] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:41 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:41.502588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe5eba-768f-4d1d-8732-62d72e63607a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:41.512364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111337ec-b429-488e-a6ff-2c98fb33fe82 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.512644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111337ec-b429-488e-a6ff-2c98fb33fe82 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.512916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111337ec-b429-488e-a6ff-2c98fb33fe82 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.514377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 496/1013] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:41.519348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.520256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.520531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.539640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:41.541104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:41.610865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 497/1014] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:41 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:41.622142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe5eba-768f-4d1d-8732-62d72e63607a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:41.633766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:41.658900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:41.659606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.659881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.660785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.661673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:41.663074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:41.672705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:41.673677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:41.676907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 518/1015] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:41.679690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:41.701917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:41.734245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:41.739739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a827038-d64e-4401-a8e1-885c8a5563ca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:41.780094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:41.781040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.781321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.781752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.783163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:41.784757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:41.786163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566db0-ffa6-49bf-9421-0e4b4531d59c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:41.787424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 498/1016] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:41.792959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:41.821687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:41.822304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.822581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.822850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.823516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:41.823813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:41.825200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:41.836192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 519/1017] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:41.837235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:41.900282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761008-2098-4bd6-babd-319dcbd2c686 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:41.937660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:41.938285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:41.938567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:41.938842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:41.939519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:41.940976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:41.945775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:41.949833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef9ee-bebf-4308-8772-84b716c3df3d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:41.979985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 499/1018] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:13:41 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:13:41.980231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:42.000103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.000760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.001087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.001366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.002062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:42.002369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.004394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ce9f-7dc8-4fa2-9857-5af8338ed92c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.012011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 520/1019] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Aug 4 10:13:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:42.016356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.028987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c6bc-55f3-46b2-8f13-58329b7e3664 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.029349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c6bc-55f3-46b2-8f13-58329b7e3664 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.029543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c6bc-55f3-46b2-8f13-58329b7e3664 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.031170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 521/1020] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:42.036730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.037047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.037285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.057835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:42.059109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:42.135370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f16eb-af07-4044-bac0-315ee92bf4da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:42.138280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 522/1021] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:42.161803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.173016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.173861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.174149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.174425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.175216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.176893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.202530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 500/1022] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:42.284884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bb50c-f51f-4bf1-8a49-a9da40e8e704 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:42.298035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:42.339988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.340774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.341163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.341446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] PATH_INFO: `/v3/services/image` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.342089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.343556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:42.354623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:42.356129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:42.357545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:42.360395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 523/1023] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Aug 4 10:13:42 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:42.362818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.413568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:42.483178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776e5de-a72b-4c3c-b2a8-0cd39272b465 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:42.527224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.527852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.528164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.528440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.529132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:42.529453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.530854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.536993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 501/1024] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:42.543066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.662913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6379c4-a301-49d8-9048-8865aaca14bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:42.707418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.708077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.708368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.708633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.709317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] RBAC: Adding query filter params (type=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:42.709621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.710991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.716662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 524/1025] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:42.722493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.827255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ffe5-e68a-49da-928e-5aa313514d52 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:42.834156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:42.862894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:42.864103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:42.864386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:42.864659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:42.865412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] RBAC: Adding query filter params (interface=public, service_id=56773525373a4b558e79c31d629c9870, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:42.865724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:42.867079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7fcf2-4ca2-4de3-b5f4-6d2beb6d9153 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:42.872411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 502/1026] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:13:42 2022] GET /identity/v3/endpoints?service_id=56773525373a4b558e79c31d629c9870&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:42.877899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:42.914296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:42.984262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3a31-74ca-4454-851b-912c9ffab890 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:43.017758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:43.018437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:43.018725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:43.018994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:43.020234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:43.021584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b38fc-cd82-4766-952d-8053a8356d7e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:43.027941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 525/1027] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:13:42 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:13:44.052773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01672a-6cda-4b70-8ee8-af38071d17c0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.056176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01672a-6cda-4b70-8ee8-af38071d17c0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.056176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01672a-6cda-4b70-8ee8-af38071d17c0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.056176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 503/1028] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:44.059847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.060354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.060509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.078620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:44.079776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:44.158294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 526/1029] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:44.183537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01672a-6cda-4b70-8ee8-af38071d17c0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:44.290713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01672a-6cda-4b70-8ee8-af38071d17c0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:44.324802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:44.325366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.325645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.325913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.327048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:44.328492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd0ad-6c86-40fb-86a2-894acec97137 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:44.334728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 504/1030] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:44.335840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:44.365515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:44.439548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c4d7-5431-4d61-8e2e-9ad4bcff5e8f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:44.473270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:44.473895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.474175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.474445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.475135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:44.476606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:44.479366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81547e-b86d-40ad-8165-44a11469e1a9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.479652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81547e-b86d-40ad-8165-44a11469e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.479927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81547e-b86d-40ad-8165-44a11469e1a9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.481411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 505/1031] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:44.485998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.486609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.486759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.500747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 527/1032] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:13:44 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:44.510657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:44.512092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:44.561156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:44.604287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 506/1033] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:44.628281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:44.719361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:44.738410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabe7a7-3397-4476-9502-14f876c6dd12 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:44.772930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:44.773576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.773856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.774130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.784247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:44.790019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:44.790987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:44.791896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:44.793279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 528/1034] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:44.797773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:44.900529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054f6e-af46-4f6b-92f2-75cfbd02598e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:44.938750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:44.939370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:44.939651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:44.939926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:44.940665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:44.940967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:44.942369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34037802-c79a-44bb-92f7-65c292893222 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:44.954974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 507/1035] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:44.956322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.059320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404491-f4fb-4794-92ba-1ef08639d085 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.070262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c2b0-d143-4c21-a32f-67f539031b92 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.070864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c2b0-d143-4c21-a32f-67f539031b92 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.070864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c2b0-d143-4c21-a32f-67f539031b92 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.072310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 508/1036] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:45.077424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.078349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.078349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.095580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.096237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:45.096358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.096605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.096830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.097581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:45.107905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.109374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.112015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.122155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:45.126271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:45.127143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:45.128291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:45.128291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:45.128291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:45.128291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:45.128291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 529/1037] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:45.135070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.173894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 509/1038] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:45.197627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.260611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f5b6-bdc8-4d16-8aae-894b793172e9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.299095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.299716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.300018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.300299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.300989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:45.301295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.302815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.303896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ee4a4-da24-41e7-8621-8b03e3465b61 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.315253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 530/1039] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:45.321064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.343191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.343824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.344128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.344397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.345011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.346362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:45.355234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:45.356279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:45.358337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 510/1040] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:45.362243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.379320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.389374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.433502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9b5-475f-4cc8-a6be-6486d8ffdb3e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.466635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901fc23-73b7-4865-924c-47153c4cda83 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.469313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.470057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.470435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.470808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.479537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.481127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:45.493943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:45.495066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:45.496118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 531/1041] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:45.500882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.505070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.505794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.506075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.506349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.507020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:45.507322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.508704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.518357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 511/1042] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:45.519541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.609061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.613822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457a2ee-29ec-4725-90f3-bc26cb2c4ec6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.623205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc869d5d-5ddf-45de-9b03-2ecf1783b4c2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.631050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.650734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.651482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.651842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.652219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.653096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:45.653475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.654971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.661609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.662337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.662687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.663031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.663785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:45.664185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.665629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.671119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 512/1043] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:45.675067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 532/1044] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:45.677879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.681470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:45.789364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc784b-1676-4efc-9a82-fc5b2d80d44d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.796301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96ae4-5f8c-4718-b9a4-49ab191f2ee1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:45.824769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.825640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.825917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.826284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.827087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.828568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:45.834607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:45.835284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:45.835499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:45.835777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:45.851550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 513/1045] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:13:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:13:45.894651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:45.925614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:45.927667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.037504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 533/1046] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:46.043941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.055882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:46.155288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1089-890a-4663-b25e-d413968e8a11 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.189530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.190177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.190554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.190836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.202203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:46.203652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:46.218130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:46.219039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:46.220038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:46.220038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:46.220038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:46.220038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:46.220038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 514/1047] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:46.225305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.304589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:46.330156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5126-968a-4290-b887-e3d120d582dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.364956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.365572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.365847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.366114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.366818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:46.367117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:46.368507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.384740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 534/1048] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:46.386005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.443923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:46.487474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c174f08-0506-4f4e-96bd-abee96ed0d56 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.521510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.522140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.522416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.522686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.529239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:46.530657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9332a8b2-45e6-4368-ae5d-83d7b782f93a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:46.542399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:46.543290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:46.544233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 515/1049] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:46.551753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.592315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68afb-325e-4ccb-bb25-dd0edb7b3498 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.592866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68afb-325e-4ccb-bb25-dd0edb7b3498 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.593335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68afb-325e-4ccb-bb25-dd0edb7b3498 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.595636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 516/1050] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:46.601069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.601759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.602670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.626582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:46.629230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:46.659150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d05a-db25-47cf-b900-83b5e18dbb1f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.694582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.695484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.695766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.696067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.696833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:46.697134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:46.698547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.700090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:46.708001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 517/1051] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:46.715993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 535/1052] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:46.717368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.724978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:46.764559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:46.823332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b83029-c256-4eb9-b4a9-31266b9ece4a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.830132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66172986-a213-432e-8a2d-2ec9a61143f1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:46.859728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.860286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.860568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.860838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.861615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] RBAC: Adding query filter params (role.id=9015a308ce9d4b9fa729fe5e1a931c21, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:46.861915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:46.864008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:46.864861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:46.865499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:46.865782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:46.866051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:46.875989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 518/1053] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=9015a308ce9d4b9fa729fe5e1a931c21 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:46.876870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:46.882092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:46.883124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:46.884188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 536/1054] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:46.889723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:47.009256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d83d-7f95-4416-9efb-484bbdc25ccc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:47.021380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:47.054103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:47.054720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.054998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.055269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.056029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:47.056339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:47.057701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:47.066577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 519/1055] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:47.072666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:47.131066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:47.177577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b14f5-b0a9-494f-b67c-ccad01c05d35 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:47.211860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:47.212529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.212812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.213094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] PATH_INFO: `/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.226413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] RBAC: Authorizing `identity:get_user(user_id=c3fb8887b02c4ef89d6db7ce2d327a0e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:47.227854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:47.229730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 537/1056] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:13:47 2022] GET /identity/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:47.235214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:47.342292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b628-e9ab-42d2-bd54-6537b349b777 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:47.376448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:47.377068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.377351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.377628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.418152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:47.446116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:47.447554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:47.551218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 520/1057] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:13:47.553024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:47.652689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ec4a0-f7b4-497d-9634-4d2ba56e7820 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:47.666188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:47.688153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:47.688584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.688867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.689141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.690285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:47.691763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0ce88-74ea-463a-8f43-b3ab1103ae13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:47.698374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 538/1058] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:13:47 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:47.699881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:47.771738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:47.803983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75b98c-4123-43e8-ba46-52fc1ae079f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:47.838351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:47.838960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.839235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.839501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.840303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7, user.id=c3fb8887b02c4ef89d6db7ce2d327a0e) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:47.840608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:47.842014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b321726-7e71-4680-a17a-7f8ae3a9387d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:47.856359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 521/1059] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:47 2022] GET /identity/v3/role_assignments?user.id=c3fb8887b02c4ef89d6db7ce2d327a0e&scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:47.909210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c7aa-f1ff-4497-b93a-3d82e922dc7a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.909589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c7aa-f1ff-4497-b93a-3d82e922dc7a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.910028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c7aa-f1ff-4497-b93a-3d82e922dc7a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.912032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 539/1060] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:47.917792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:47.918091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:47.918349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:47.938941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:47.940137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:48.019983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 522/1061] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:48.039393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95c7aa-f1ff-4497-b93a-3d82e922dc7a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:48.145096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95c7aa-f1ff-4497-b93a-3d82e922dc7a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:48.179335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:48.179971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:48.180258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:48.180533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:48.228340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:48.254940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:48.256426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:48.355848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 540/1062] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:13:48.361537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:48.426933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:48.462929 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e40b3-7295-4d83-a0bd-019ac40fd4ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:48.497624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:48.498190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:48.498468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:48.498738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:48.509422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:48.510854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b98e53eb-bfc3-4d23-b2f2-9a9764f5d455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:48.525434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:48.526354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:48.527231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:48.527231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:48.527231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:48.527231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:48.528018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 523/1063] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:48.532533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:48.620533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:48.646531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85305f7-8811-47ee-b8e1-894583d2f76c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:48.687506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:48.688150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:48.688432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:48.688702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:48.689381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:48.689681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:48.691040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c956c-1d98-4747-ab43-cad583e33c4b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:48.702724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 541/1064] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:48.900265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c4c6-d0f8-4957-a941-bfaa3f13570c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:48.900638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c4c6-d0f8-4957-a941-bfaa3f13570c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:48.900859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c4c6-d0f8-4957-a941-bfaa3f13570c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:48.902186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 524/1065] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:48.907351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:48.907649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:48.907931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:48.927803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:48.929004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:49.001836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 542/1066] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:49.023759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c4c6-d0f8-4957-a941-bfaa3f13570c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.124055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c4c6-d0f8-4957-a941-bfaa3f13570c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.140358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:49.157520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.158081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.158361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.158634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.170350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:49.175463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:49.176421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:49.177271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 525/1067] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:49.182620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.249420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:49.280317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0ff65-a1ac-44d7-b956-5edf4b08163f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.314620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.315234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.315514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.315784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.316569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:49.316872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:49.318243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:49.326141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 543/1068] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:49.332478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.428953 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf9a04-4538-4300-a1e4-ea1719f3f4e2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.461487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.462125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.462408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.462684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.473469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:49.474902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:49.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:49.489658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:49.490534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:49.490534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:49.490534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:49.490534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:49.490534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 526/1069] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:49.495488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.594583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdff1-7cac-4082-a6b4-cc1b2b8382c2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.623142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:49.628883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.629491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.629769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.630039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.630870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:49.631170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:49.632554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:49.644119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 544/1070] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:49.649617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.680550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:49.749342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2464-70fc-492d-9afe-fd52063c08a0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.781166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.781795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.782075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.782345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.790188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:49.791610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c3b5ec22-e01b-429c-9ae0-668c4c7913bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:49.803385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:49.804526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:49.810220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 527/1071] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:49.810797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.875432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc946d-0fd5-4cf9-828d-a97cc7bda010 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.875709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc946d-0fd5-4cf9-828d-a97cc7bda010 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.876014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc946d-0fd5-4cf9-828d-a97cc7bda010 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.877351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 528/1072] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:49.882212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.882496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.882769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.901030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:49.902215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:49.915653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad822-cf56-49bb-9055-e951572e055d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:49.947909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:49.948479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:49.948773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:49.949021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:49.949805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:49.950109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:49.951527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:49.968612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 545/1073] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:49.971653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 529/1074] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:49.974832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:49.997542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.018281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.025470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.074719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9613ee2-7a08-4cc2-992c-0e2362fefe87 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.098755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54160c62-c14d-4f81-96c8-1de723e3e8e7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.108423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.109062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.109344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.109614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.119519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.120987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.131638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.132306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.132589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.132863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:50.135159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:50.136219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:50.137163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:50.137163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:50.137163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:50.137163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:50.137163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 546/1075] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:50.141876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.147408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:50.152377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:50.153523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:50.154571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 530/1076] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:50.159394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.242327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.249880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4e60-4702-497a-ace8-0d34f808977a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.251477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.263445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d452ab-de41-4372-8a83-22dc0280ae37 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.285065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.285681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.285963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.286234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.286917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:50.287221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.288616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.297418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.298021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.298288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.298559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.299253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:50.299696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.301083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.301659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 547/1077] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:50.307205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.313602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 531/1078] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:50.314550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.407330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bebd4-b8c3-4c64-9916-5bef41362bc4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.416202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881d1ec-13ed-444b-987e-e6c851ad430d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.440418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.441046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.441834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.442099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.448513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.449922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.450244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.450871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.451150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.451422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] PATH_INFO: `/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.452605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] RBAC: Authorizing `identity:get_user(user_id=c3fb8887b02c4ef89d6db7ce2d327a0e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.454004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.455588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 532/1079] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:50.462180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:50.462311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:50.463178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:50.464122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 548/1080] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:50.468898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.474715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.493547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.563170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776ba0-948c-45b3-ac4b-343a9524a299 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.569682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f5cd-1bdb-461f-bb8c-bef938104ac8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.597282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.598492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.598768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.599047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.600221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.601640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.602903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.603272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 533/1081] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:50.603514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.603797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.604116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.604865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:50.605171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.606581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.609268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.619788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 549/1082] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:50.625876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:50.685692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.704514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:50.707760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff887e-2dbe-4a71-bc73-4e9799dc8f79 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.723686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634af47e-14c1-487f-9f6b-96d414ac4c79 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:50.740178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.740833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.741110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.741385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7/users/c3fb8887b02c4ef89d6db7ce2d327a0e/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.743303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7, user_id=c3fb8887b02c4ef89d6db7ce2d327a0e, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.744801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f5ffb-5ec2-43a7-a0c5-35bef012bb61 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.756908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:50.757160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 534/1083] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:50 2022] PUT /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7/users/c3fb8887b02c4ef89d6db7ce2d327a0e/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:13:50.767144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:50.767144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:50.767144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/9015a308ce9d4b9fa729fe5e1a931c21` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:50.767144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=45bdd938403d46d6b7b026abd46c564d, role_id=9015a308ce9d4b9fa729fe5e1a931c21)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:50.767144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fdf5a-1321-4a8d-bb65-252b374f2191 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:50.778121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 550/1084] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:13:50 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/45bdd938403d46d6b7b026abd46c564d/roles/9015a308ce9d4b9fa729fe5e1a931c21 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:13:52.755584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355cc9-c2d0-4555-919b-78c9ca7f9579 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:52.756149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355cc9-c2d0-4555-919b-78c9ca7f9579 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:52.756236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355cc9-c2d0-4555-919b-78c9ca7f9579 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:52.757715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 535/1085] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:52.762796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:52.763241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:52.763335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:52.772015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10610-6454-4f0e-8d13-fd4e27c3a567 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:52.772306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10610-6454-4f0e-8d13-fd4e27c3a567 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:52.772581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10610-6454-4f0e-8d13-fd4e27c3a567 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:52.773877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 536/1086] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:13:52.778786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:52.779073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:52.779349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:52.781090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:52.782290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:52.798451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:52.799606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:52.849590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 551/1087] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:52.865372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 537/1088] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:13:52.873024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:52.888474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:52.941683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:52.954838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:52.972108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210ca6-0021-4a55-b7e4-88457ff3a8e0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:52.993456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0431-e009-4fc5-9635-5298dfb690e9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.008373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.009020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.009292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.009557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.020050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:53.024910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:53.025968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:53.026808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 552/1089] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:53.029856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.030505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.030785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.031059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.032344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.042426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:13:53.047593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:53.048603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:53.049419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 538/1090] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Aug 4 10:13:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:53.055192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.130920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb334-109f-4cbc-b134-b9c87edd29a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.152157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b45c2-f9d6-4f34-91ea-5767b7730bdb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.158437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.164416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.165028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.165305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.165578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.166289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:53.166590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.167964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.176855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 553/1091] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.180000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.183273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.188364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.188974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.189249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.189520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.190240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:53.190539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.191881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.205428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 539/1092] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.206426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.285781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b5288-2c45-4b32-9699-37e109e85054 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.307158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ced81a-9d1f-49b2-b68f-c3f69005d07c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.321831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.322476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.322759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.323035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] PATH_INFO: `/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.324253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] RBAC: Authorizing `identity:get_user(user_id=c3fb8887b02c4ef89d6db7ce2d327a0e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.325801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.327409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 554/1093] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/users/c3fb8887b02c4ef89d6db7ce2d327a0e => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.333648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.344845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.345805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.346245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.346537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.358139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.359586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:53.373856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:53.374721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:53.375641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:53.375641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:53.375641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:53.375641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:53.375641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 540/1094] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:53.381171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.396066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.424113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.434364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695a09-23e6-430d-9d80-753fb08cac63 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.468800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.469416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.469692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.469962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.481016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873016f-c076-4894-b2ba-da0c132bb4ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.514619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.515229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.515511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.515785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.516501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:53.516804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.518151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.530359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 541/1095] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:53.535921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.536112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.537651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.596894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.634401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682815e-a024-4a25-a03c-597cd4eec59e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.635646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 555/1096] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:53.636188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.641497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.672363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.672999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.673282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.673560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.680955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.682380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:13:53.694377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:53.695480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:13:53.696523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 542/1097] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:13:53.701199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.742607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c9f2e-93d8-4d76-9d17-6aaf2795c9b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.777149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.777791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.778052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.778325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.779491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.780962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.782793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 556/1098] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.788618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.797830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b90048-8174-4430-9be5-3696606d93bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.818001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.832500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.833112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.833392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.833658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.834439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:53.834740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.836170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.852583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 543/1099] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.858350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:53.880164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:53.887273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ded22-dab0-4bf9-a31e-969208f87887 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.920396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.921000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.921273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.921540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:53.922571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7, user.id=c3fb8887b02c4ef89d6db7ce2d327a0e) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:53.922873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:53.924307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:53.931091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 557/1100] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/role_assignments?user.id=c3fb8887b02c4ef89d6db7ce2d327a0e&scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:13:53.959187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39387e-bbbd-44d4-82c5-4ee9280bf18b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:53.995746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:53.996411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:53.996692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:53.996965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.072057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.073499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.095905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:54.173362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 544/1101] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:13:54.178983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:54.275551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:54.282759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4fe7-7243-468d-b94e-1123c11533a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:54.314997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:54.315628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:54.315909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:54.316737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.327043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.328451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:13:54.343093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:54.344069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:54.344966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:13:54.344966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:13:54.344966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:13:54.344966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:54.344966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 558/1102] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:54.349931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:54.448673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9266-2004-4277-ac7e-56702e4d6d80 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:54.467826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:54.482011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:54.482624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:54.482901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:54.483171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.483854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:54.484183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.485526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.496978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 545/1103] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:54.502580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:54.599724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60dfdb-f383-409a-9fe5-4d5086bdddd1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:54.631395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:54.632055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:54.632336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:54.632606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.639066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.640615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:13:54.652667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:13:54.653729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:13:54.654646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 559/1104] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:13:54.659673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:54.761100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c77ae-58e0-4ee5-b007-4d5b3aa8dbbe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:54.794345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:54.794954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:54.795234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:54.795504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.796279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:54.796584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.798252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef0d3e-4080-4ef1-8649-a772adb3fb43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.809826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 546/1105] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:54.815923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:54.825584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:54.913287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c7fe-96c0-46c8-a242-ad0927bbae02 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:54.945993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:54.946600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:54.946879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:54.947152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:54.947942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] RBAC: Adding query filter params (role.id=9015a308ce9d4b9fa729fe5e1a931c21, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=45bdd938403d46d6b7b026abd46c564d) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:54.948276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:54.949698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8de4-7010-4975-9b03-dad1531c13a2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:54.955275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 560/1106] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:54 2022] GET /identity/v3/role_assignments?user.id=45bdd938403d46d6b7b026abd46c564d&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=9015a308ce9d4b9fa729fe5e1a931c21 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:55.997434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915a520-a447-4ccf-a7d9-ac98e55da768 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:55.997968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915a520-a447-4ccf-a7d9-ac98e55da768 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:55.998059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915a520-a447-4ccf-a7d9-ac98e55da768 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:55.999457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 547/1107] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:56.004666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:56.005108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:56.005201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:56.023620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:56.024802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:56.085027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:56.089690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 561/1108] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:56.114277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915a520-a447-4ccf-a7d9-ac98e55da768 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:56.155816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:56.214048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915a520-a447-4ccf-a7d9-ac98e55da768 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:56.247291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:56.247965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:56.248279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:56.248554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:56.249705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:56.251160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f1ff-9f54-4c8f-b6c5-2f1faaea428d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:56.252821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 548/1109] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:56.258960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:56.359056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527c783-934f-4d9b-8f5d-0b64d1a13a5b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:56.391135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:56.391731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:56.392030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:56.392301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:56.392964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:56.395130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb248981-8f5e-46d5-851a-627566423130 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:56.415684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 562/1110] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:13:56 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:13:58.468431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8a5e7-c14a-4de8-b79f-68700e322c67 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:58.468947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8a5e7-c14a-4de8-b79f-68700e322c67 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:58.468947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8a5e7-c14a-4de8-b79f-68700e322c67 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:58.470307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 549/1111] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:13:58.475436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268d50-7889-4b07-be4f-98399515425d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:58.475831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268d50-7889-4b07-be4f-98399515425d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:58.476010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268d50-7889-4b07-be4f-98399515425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:58.494199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9a268d50-7889-4b07-be4f-98399515425d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:13:58.495373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9a268d50-7889-4b07-be4f-98399515425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:13:58.543426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:58.560721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 563/1112] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:13:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:13:58.583680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8a5e7-c14a-4de8-b79f-68700e322c67 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:58.681317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8a5e7-c14a-4de8-b79f-68700e322c67 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:58.689954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:58.713467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:58.714091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:58.714374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:58.714647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] PATH_INFO: `/v3/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:58.716119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] RBAC: Authorizing `identity:get_role(role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:58.717486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:58.718800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 550/1113] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:13:58 2022] GET /identity/v3/roles/9a068445d06b48629659d5184c3e91bc => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:58.724997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268d50-7889-4b07-be4f-98399515425d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:58.821328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268d50-7889-4b07-be4f-98399515425d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:58.854708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:58.855335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:58.855612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:58.855880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] PATH_INFO: `/v3/users/a00053bdbd6d4b9b8f748a664bbea836` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:58.868940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] RBAC: Authorizing `identity:get_user(user_id=a00053bdbd6d4b9b8f748a664bbea836)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:58.870372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:58.871972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 564/1114] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:13:58 2022] GET /identity/v3/users/a00053bdbd6d4b9b8f748a664bbea836 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:58.877737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:58.974320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f5c1c-f9c8-4604-a580-10c949f71749 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:59.008021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:59.008632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:59.008916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:59.009191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:59.052332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:59.075450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:59.076886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:59.174871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 551/1115] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:13:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:13:59.180827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:59.225147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:59.279602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd542c-9290-48aa-a4c8-7b0dcab5aa09 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:59.312094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:59.312725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:59.313005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:59.313278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:59.314415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:59.315910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334e26-ac7f-417b-82d9-c2413aa97c32 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:59.318061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 565/1116] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:13:59 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:13:59.324284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:13:59.398318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:13:59.420897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112381-b32b-4a3f-8710-9a234c769acf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:13:59.462362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:13:59.462973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:13:59.463251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:13:59.463527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:13:59.464338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7, user.id=a00053bdbd6d4b9b8f748a664bbea836) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:13:59.464647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:13:59.466062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec773730-15cd-40ff-b1b0-8efc71ad8734 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:13:59.471258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 552/1117] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:13:59 2022] GET /identity/v3/role_assignments?user.id=a00053bdbd6d4b9b8f748a664bbea836&scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7&role.id=9a068445d06b48629659d5184c3e91bc => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:01.423255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc3cf-e146-4e7a-9468-493d57bb7bda None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:01.423804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc3cf-e146-4e7a-9468-493d57bb7bda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:01.423804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc3cf-e146-4e7a-9468-493d57bb7bda None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:01.425199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 566/1118] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:01.430867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:01.431271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:01.431420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:01.449578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:01.450775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:01.514292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 553/1119] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:01.536626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc3cf-e146-4e7a-9468-493d57bb7bda None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:01.633600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc3cf-e146-4e7a-9468-493d57bb7bda None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:01.645970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:01.666388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:01.667005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:01.667286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:01.667555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] PATH_INFO: `/v3/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:01.668979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] RBAC: Authorizing `identity:get_role(role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:01.670320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:01.671617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 567/1120] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:01 2022] GET /identity/v3/roles/9a068445d06b48629659d5184c3e91bc => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:01.678308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:01.744155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:01.773399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338858c9-c278-4cdf-80c4-da2f13d1a0fc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:01.806554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:01.807189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:01.807471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:01.807744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] PATH_INFO: `/v3/users/a00053bdbd6d4b9b8f748a664bbea836` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:01.808904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] RBAC: Authorizing `identity:get_user(user_id=a00053bdbd6d4b9b8f748a664bbea836)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:01.810355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:01.811918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 554/1121] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:01 2022] GET /identity/v3/users/a00053bdbd6d4b9b8f748a664bbea836 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:01.817649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:01.918790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f496c1-adfe-48d4-ab86-15604cd590a7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:01.950546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:01.951164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:01.951440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:01.951705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:01.952847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:01.954266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993f8f9-7996-477c-9fd9-d6a42bcff383 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:01.956025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 568/1122] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:01 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:01.962398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:02.057316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ac92b-698c-4d5d-9c98-7249e343aa1e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:02.089928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:02.090578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:02.090856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:02.091128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7/users/a00053bdbd6d4b9b8f748a664bbea836/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:02.093302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7, user_id=a00053bdbd6d4b9b8f748a664bbea836, role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:02.094730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758f45e-b4e5-4a7e-9406-7cb1a06b9063 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:02.103429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:02.109685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 555/1123] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:14:01 2022] PUT /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7/users/a00053bdbd6d4b9b8f748a664bbea836/roles/9a068445d06b48629659d5184c3e91bc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:14:04.083979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febacdba-5a5e-4825-a136-aa1b57821f27 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.084537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febacdba-5a5e-4825-a136-aa1b57821f27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.084537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febacdba-5a5e-4825-a136-aa1b57821f27 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.085854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 569/1124] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:04.090925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.091301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.091487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.109073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:04.110286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:04.174166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 556/1125] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:04.198011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febacdba-5a5e-4825-a136-aa1b57821f27 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:04.246938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:04.296663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febacdba-5a5e-4825-a136-aa1b57821f27 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:04.329691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:04.330325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.330607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.330879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] PATH_INFO: `/v3/roles/9a068445d06b48629659d5184c3e91bc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.332331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] RBAC: Authorizing `identity:get_role(role_id=9a068445d06b48629659d5184c3e91bc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:04.333690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:04.335073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 570/1126] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:04 2022] GET /identity/v3/roles/9a068445d06b48629659d5184c3e91bc => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:04.340755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:04.436616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbc260-6825-4f89-8704-9f2c2596e690 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:04.462112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:04.470143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:04.470773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.471054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.471354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] PATH_INFO: `/v3/users/a00053bdbd6d4b9b8f748a664bbea836` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.472536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] RBAC: Authorizing `identity:get_user(user_id=a00053bdbd6d4b9b8f748a664bbea836)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:04.473967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:04.475551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 557/1127] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:04 2022] GET /identity/v3/users/a00053bdbd6d4b9b8f748a664bbea836 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:04.481443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:04.577902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed851b-51cb-4f68-8d3d-d7e8cb6d637e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:04.593512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:04.611075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:04.611674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.612072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.612349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.676668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:04.678093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:04.770260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 571/1128] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:14:04.776273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:04.870984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c519e3-167e-43a9-8c4d-4fcf4c5af6cd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:04.903512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:04.904168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:04.904447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:04.904720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] PATH_INFO: `/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:04.905849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] RBAC: Authorizing `identity:get_project(project_id=e5e59ab0b33f420f8b9f3baf5ad021f7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:04.907264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a133d71-6260-4c3b-8671-a4a1333b5e5c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:04.908987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 558/1129] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:04 2022] GET /identity/v3/projects/e5e59ab0b33f420f8b9f3baf5ad021f7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:04.915575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:04.932644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:05.011745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c18f5-062d-4789-8fe5-0cfb75d35e62 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:05.044322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:05.044914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:05.045189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:05.045454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:05.046226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] RBAC: Adding query filter params (role.id=9a068445d06b48629659d5184c3e91bc, scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7, user.id=a00053bdbd6d4b9b8f748a664bbea836) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:05.046527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:05.048223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274971-7d82-49a6-b7ad-5078d9e3cc96 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:05.053783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 572/1130] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:14:04 2022] GET /identity/v3/role_assignments?user.id=a00053bdbd6d4b9b8f748a664bbea836&scope.project.id=e5e59ab0b33f420f8b9f3baf5ad021f7&role.id=9a068445d06b48629659d5184c3e91bc => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:07.053982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901cf07-9435-4398-a533-546c16ab0a8d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:07.054511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901cf07-9435-4398-a533-546c16ab0a8d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:07.054511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901cf07-9435-4398-a533-546c16ab0a8d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:07.055838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 559/1131] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:07.060913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:07.061354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:07.061463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:07.079334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:07.080532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:07.144031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 573/1132] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:07.167045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901cf07-9435-4398-a533-546c16ab0a8d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:07.227419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:07.267274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901cf07-9435-4398-a533-546c16ab0a8d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:07.299619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:07.300268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:07.301047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:07.301319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:07.302457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:07.303861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67409578-bcac-49bf-a314-fd55bc4a0465 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:07.305513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 560/1133] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:07.311163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:07.322946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:07.405856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40aea6-2b89-44c8-9c4d-9879b1ddccbc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:07.438158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:07.438757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:07.439034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:07.439304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:07.440009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:07.442015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc76806-8c18-43f9-be7e-4f703ba58e6c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:07.467745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 574/1134] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:14:07 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:14:09.481708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb4b5-bc82-4911-8075-88baede7b990 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:09.482226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb4b5-bc82-4911-8075-88baede7b990 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:09.482334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb4b5-bc82-4911-8075-88baede7b990 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:09.483578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 561/1135] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:09.488774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:09.489154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:09.489331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:09.506783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:09.507893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:09.562714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:09.573547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 575/1136] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:09.599151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb4b5-bc82-4911-8075-88baede7b990 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:09.697435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb4b5-bc82-4911-8075-88baede7b990 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:09.726150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:09.731484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:09.732140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:09.732424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:09.732699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:09.733844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:09.735264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf32f9-1e23-43ca-ab60-51bf953c3b82 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:09.736849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 562/1137] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:09.742531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:09.835661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38766ac1-7785-4b34-9fd8-2b2bc4e19d96 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:09.866833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:09.867427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:09.867702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:09.867994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:09.868657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:09.870053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95c441-808a-4974-bf36-900e04baa383 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:09.891385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 576/1138] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:14:09 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:14:11.916156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e990f3-3999-4cef-bdd8-61923b1540fb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:11.916705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e990f3-3999-4cef-bdd8-61923b1540fb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:11.916705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e990f3-3999-4cef-bdd8-61923b1540fb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:11.918013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 563/1139] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:11.923460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:11.923852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:11.924033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:11.942948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:11.944165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:11.986438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:12.010306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 577/1140] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:12.032778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e990f3-3999-4cef-bdd8-61923b1540fb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:12.130538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e990f3-3999-4cef-bdd8-61923b1540fb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:12.164112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:12.164734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:12.165014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:12.165292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:12.175253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:12.181032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:12.182240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:12.183332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 564/1141] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:12.188254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:12.284589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc04b-3254-46dd-8594-b2d8544e8af9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:12.317384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:12.317992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:12.318269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:12.318537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:12.319240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:12.319541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:12.320926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:12.328591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 578/1142] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:12.334303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:12.391093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:12.432481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfaebe-260f-4ec0-99bd-23786c5878f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:12.464408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:12.465031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:12.465310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:12.465582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] PATH_INFO: `/v3/users/9b94cb7e5456478a9daa6d8f8b131b67` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:12.478719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] RBAC: Authorizing `identity:get_user(user_id=9b94cb7e5456478a9daa6d8f8b131b67)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:12.480192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:12.481780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 565/1143] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/users/9b94cb7e5456478a9daa6d8f8b131b67 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:12.487558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:12.528880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:12.585711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde6d5-2e3d-4d8e-b649-6737410231fe None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:12.617268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:12.617866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:12.618139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:12.618403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:12.684025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:12.685426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:12.723748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:12.779815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 579/1144] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:14:12.785830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:12.880922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2f0b4-daab-4254-b6a0-2f5452a5fe72 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:12.897801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:12.914405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:12.915035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:12.915312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:12.915589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] PATH_INFO: `/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:12.916754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] RBAC: Authorizing `identity:get_project(project_id=a08fb7fd4d3e42698ef761bcc71e4676)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:12.918187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cea4e-1aa3-441c-bee8-a030595f8a04 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:12.919818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 566/1145] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:14:12.926061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:13.019870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89696-53f2-41d2-9141-d689ca400338 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:13.052087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:13.052678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:13.052955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:13.053228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:13.054000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=a08fb7fd4d3e42698ef761bcc71e4676, user.id=9b94cb7e5456478a9daa6d8f8b131b67) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:13.054306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:13.055711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3abc4-4aa9-486a-838a-8a1931b0ec35 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:13.060940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 580/1146] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:14:12 2022] GET /identity/v3/role_assignments?user.id=9b94cb7e5456478a9daa6d8f8b131b67&scope.project.id=a08fb7fd4d3e42698ef761bcc71e4676&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:15.003516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b240-3725-495d-a3cb-5c7a87bcfee0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.004751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b240-3725-495d-a3cb-5c7a87bcfee0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.004845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b240-3725-495d-a3cb-5c7a87bcfee0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.006187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 567/1147] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:15.011296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.011684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.011842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.018941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:15.029796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:15.030958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:15.094209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 581/1148] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:15.116515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b240-3725-495d-a3cb-5c7a87bcfee0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:15.213446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b240-3725-495d-a3cb-5c7a87bcfee0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:15.246360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:15.246983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.247257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.247530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.257444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:15.262200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:15.263104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:15.263942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 568/1149] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:15.269365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:15.363871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671f93-d8a1-44c5-82b1-2e0fa179b800 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:15.398002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:15.398603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.398879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.399154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.399857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:15.400193 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:15.401549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:15.402833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:15.410621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 582/1150] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:15.416564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:15.462080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:15.513411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a565cb-62b5-4901-9203-1443e7340be1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:15.546122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:15.546735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.547013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.547290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] PATH_INFO: `/v3/users/9b94cb7e5456478a9daa6d8f8b131b67` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.548715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] RBAC: Authorizing `identity:get_user(user_id=9b94cb7e5456478a9daa6d8f8b131b67)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:15.550115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:15.551678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 569/1151] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:15 2022] GET /identity/v3/users/9b94cb7e5456478a9daa6d8f8b131b67 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:15.557284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:15.652228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a59c4-6b06-4b11-8c9e-f7bdc83905d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:15.684483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:15.685107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.685389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.685660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] PATH_INFO: `/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.686791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] RBAC: Authorizing `identity:get_project(project_id=a08fb7fd4d3e42698ef761bcc71e4676)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:15.688222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e7005-f632-454a-b69a-25e5658c117e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:15.689849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 583/1152] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:15 2022] GET /identity/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:15.695570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:15.791476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6913a-2d29-4060-a492-442790dcd873 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:15.805516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:15.825459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:15.826110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:15.826389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:15.826662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] PATH_INFO: `/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676/users/9b94cb7e5456478a9daa6d8f8b131b67/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:15.828597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] RBAC: Authorizing `identity:create_grant(project_id=a08fb7fd4d3e42698ef761bcc71e4676, user_id=9b94cb7e5456478a9daa6d8f8b131b67, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:15.830008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fa3c5-8bcd-4d1c-bd4b-a59967424e8f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:15.842742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 570/1153] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:14:15 2022] PUT /identity/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676/users/9b94cb7e5456478a9daa6d8f8b131b67/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:14:17.842192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b447-013a-4b2f-b4b7-44f6dcb0c0d8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:17.842744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b447-013a-4b2f-b4b7-44f6dcb0c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:17.842744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b447-013a-4b2f-b4b7-44f6dcb0c0d8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:17.844119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 584/1154] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:17.849988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:17.850423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:17.850624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:17.868511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:17.869700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:17.933588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 571/1155] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:14:17.955926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b447-013a-4b2f-b4b7-44f6dcb0c0d8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.017673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:18.055143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b447-013a-4b2f-b4b7-44f6dcb0c0d8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.087200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.087817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.088125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.088399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.098464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:18.103272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:18.104378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:14:18.105379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 585/1156] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:18.110422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.209770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac49698-7fa0-4f0f-a98b-2fa4022a677a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.212171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:18.244337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.244954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.245237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.245511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.246225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:18.246529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:18.247886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:18.256690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 572/1157] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:14:18.262364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.357435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c52746-4840-44da-8834-52f25cc49c60 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.379198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:18.389709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.390333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.390608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.390872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] PATH_INFO: `/v3/users/9b94cb7e5456478a9daa6d8f8b131b67` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.392012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] RBAC: Authorizing `identity:get_user(user_id=9b94cb7e5456478a9daa6d8f8b131b67)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:18.393430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:18.395526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 586/1158] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:18 2022] GET /identity/v3/users/9b94cb7e5456478a9daa6d8f8b131b67 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:18.401363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.497187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841752-0d44-4127-a55e-e82e87db61ad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.529356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.529952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.530232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.530506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.562377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:18.602031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:18.603455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:18.697962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 573/1159] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:14:18.703970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.802156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57221034-2512-471c-aef8-e735cde62b61 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.833888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.834514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.834794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.835070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] PATH_INFO: `/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] RBAC: Authorizing `identity:get_project(project_id=a08fb7fd4d3e42698ef761bcc71e4676)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:18.837662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b74c-428d-43c5-b53d-c756d428e08c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:18.839283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 587/1160] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:18 2022] GET /identity/v3/projects/a08fb7fd4d3e42698ef761bcc71e4676 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:14:18.845388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:18.906640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:18.940468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9cc3-af2a-4b85-98f4-18c7c54bb8bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:18.973025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:18.973562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:18.973835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:18.974105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:18.974872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=a08fb7fd4d3e42698ef761bcc71e4676, user.id=9b94cb7e5456478a9daa6d8f8b131b67) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:18.975179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:18.976615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561099-c6a3-4189-b384-09086477f3b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:18.982228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 574/1161] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:14:18 2022] GET /identity/v3/role_assignments?user.id=9b94cb7e5456478a9daa6d8f8b131b67&scope.project.id=a08fb7fd4d3e42698ef761bcc71e4676&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:20.985724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d24bd-f0d6-4df7-a784-4f3ced38f370 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:20.986291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d24bd-f0d6-4df7-a784-4f3ced38f370 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:20.986291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d24bd-f0d6-4df7-a784-4f3ced38f370 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:20.987603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 588/1162] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:20.993331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:20.993772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:20.993866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:21.012725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:21.013909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:21.077578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 575/1163] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:21.099962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d24bd-f0d6-4df7-a784-4f3ced38f370 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:21.158454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:21.199428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d24bd-f0d6-4df7-a784-4f3ced38f370 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:21.231577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:21.232204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:21.232487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:21.232759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:21.299819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:21.301900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:21.356149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:21.396855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 589/1164] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:14:21.403062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:21.460734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:21.501282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f975628-0c8e-4dfe-a992-812fc0051406 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:21.534010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:21.534637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:21.535455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:21.535729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:21.542171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:21.543606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4c21592f-49d0-4550-89ee-ffcd5b621564 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:21.555028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:21.556255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:21.557340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 576/1165] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:14:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:21.562145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:21.657609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c786-7848-4f22-8412-7e180e2308e3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:21.690562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:21.691718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:21.692030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:21.692305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:21.692981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:21.693282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:21.694640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e4dcc-e67f-49f1-9289-d3e86f3cd29f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:21.701398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 590/1166] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Aug 4 10:14:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:23.684574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cb65-190f-4ba7-9a61-4e17789e46be None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:23.685121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cb65-190f-4ba7-9a61-4e17789e46be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:23.685121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cb65-190f-4ba7-9a61-4e17789e46be None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:23.686466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 577/1167] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:23.691723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57fa2-0761-46eb-bd5e-38d170baf94d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:23.692131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57fa2-0761-46eb-bd5e-38d170baf94d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:23.692295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57fa2-0761-46eb-bd5e-38d170baf94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:23.710328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a3b57fa2-0761-46eb-bd5e-38d170baf94d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:23.711495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3b57fa2-0761-46eb-bd5e-38d170baf94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:23.729102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:23.777223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 591/1168] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:23.800580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cb65-190f-4ba7-9a61-4e17789e46be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:23.897185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cb65-190f-4ba7-9a61-4e17789e46be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:23.916633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:23.930310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:23.930900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:23.931176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:23.931443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:23.932183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:23.933527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed90322-98f0-4f67-883f-5bff346dcdee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:23.948651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 578/1169] 10.222.0.72 () {60 vars in 1138 bytes} [Thu Aug 4 10:14:23 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:14:25.974658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4cdf9-4c6a-4f89-ba27-35c6157a887f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:25.975187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4cdf9-4c6a-4f89-ba27-35c6157a887f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:25.975187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4cdf9-4c6a-4f89-ba27-35c6157a887f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:25.976576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 592/1170] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:25.981588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:25.982009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:25.982129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:26.000052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:26.001206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:26.064829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 579/1171] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:26.088592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4cdf9-4c6a-4f89-ba27-35c6157a887f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:26.188002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4cdf9-4c6a-4f89-ba27-35c6157a887f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:26.221592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:26.222322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:26.222604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:26.222877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] PATH_INFO: `/v3/domains/dcd014dfe12843f4b6e99cbbd10d5dd7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:26.232894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcd014dfe12843f4b6e99cbbd10d5dd7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:26.234276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3a87f-a8f7-44a9-aad6-aa36fbb70589 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:26.236016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 593/1172] 10.222.0.72 () {56 vars in 1183 bytes} [Thu Aug 4 10:14:26 2022] GET /identity/v3/domains/dcd014dfe12843f4b6e99cbbd10d5dd7 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:26.242420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:26.336943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc8a47-e513-44e6-8190-ac678797b197 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:26.347081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:26.371067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:26.371669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:26.371946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:26.372243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:26.372924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:26.374339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721dad-ad97-4d06-99f7-1e9b8cbfe454 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:26.399520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 580/1173] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:14:26 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:14:28.439435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0db7a-bba5-43ae-9ed3-d2fdb2545de4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:28.439996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0db7a-bba5-43ae-9ed3-d2fdb2545de4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:28.439996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0db7a-bba5-43ae-9ed3-d2fdb2545de4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:28.441347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 594/1174] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:28.446502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:28.446904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:28.447050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:28.465171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:28.466376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:28.531915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 581/1175] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:14:28.555785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0db7a-bba5-43ae-9ed3-d2fdb2545de4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:28.571906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:28.656128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0db7a-bba5-43ae-9ed3-d2fdb2545de4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:28.687364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:28.688008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:28.688288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:28.688558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] PATH_INFO: `/v3/domains/dcd014dfe12843f4b6e99cbbd10d5dd7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:28.689684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcd014dfe12843f4b6e99cbbd10d5dd7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:28.691642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba53763-d8c0-4be9-bc99-70ae5cd62927 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:28.693231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 595/1176] 10.222.0.72 () {56 vars in 1183 bytes} [Thu Aug 4 10:14:28 2022] GET /identity/v3/domains/dcd014dfe12843f4b6e99cbbd10d5dd7 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:28.698903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:28.787290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:28.796760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30353f6-01fa-4c42-874c-741f63d1e495 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:28.829436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:28.830040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:28.830319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:28.830592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:28.831272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:28.833394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353819cb-ad23-4ca8-a065-f9dfc8b3b01b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:28.853619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 582/1177] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:14:28 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:14:30.890994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559e19-5c5b-4bc3-adc7-9d4f73c7c638 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:30.891581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559e19-5c5b-4bc3-adc7-9d4f73c7c638 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:30.891581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559e19-5c5b-4bc3-adc7-9d4f73c7c638 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:30.892933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 596/1178] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:30.898123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:30.898576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:30.898673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:30.916885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:30.918077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:30.981491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 583/1179] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:31.005464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13559e19-5c5b-4bc3-adc7-9d4f73c7c638 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.085079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:31.103690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13559e19-5c5b-4bc3-adc7-9d4f73c7c638 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:31.135625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:31.136280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:31.136562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:31.136835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:31.147686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:31.152527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:31.153674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:14:31.154741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 597/1180] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:31.159661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.234189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:31.257388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e30e6-cacf-49f0-972f-9a149b140e17 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:31.290769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:31.291374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:31.291654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:31.291927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:31.292664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:31.292966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:31.294319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:31.302436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 584/1181] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:31.308239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.404376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d99d9-705a-46d8-b36a-6d7c953a1a0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:31.436895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:31.437519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:31.437801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:31.438075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] PATH_INFO: `/v3/users/fa378b7521b24f4fb4cca0dfa53cd969` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:31.448677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:31.451216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] RBAC: Authorizing `identity:get_user(user_id=fa378b7521b24f4fb4cca0dfa53cd969)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:31.452909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:31.454694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 598/1182] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/users/fa378b7521b24f4fb4cca0dfa53cd969 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:31.460952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.557800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1766c-9a98-4385-a200-305f0753b371 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:31.590060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:31.590654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:31.590931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:31.591201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:31.601538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:31.660296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:31.661653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:31.754447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 585/1183] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:14:31.760542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.856568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f426d-1abc-4549-ac4f-1f225de8c6cc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:31.888719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:31.889342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:31.889621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:31.889894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] PATH_INFO: `/v3/projects/c2e6d91256e54c4493211d86d75e0ca2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:31.891025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e6d91256e54c4493211d86d75e0ca2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:31.892473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781126cc-3baf-4ff6-bd3c-80f1f30bf855 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:31.894597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 599/1184] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/projects/c2e6d91256e54c4493211d86d75e0ca2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:31.900543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:31.953847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:32.017125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89d6db-882c-4ed7-9fb6-33b9bb31f096 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:32.050242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:32.050871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:32.051151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:32.051424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:32.052230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=c2e6d91256e54c4493211d86d75e0ca2, user.id=fa378b7521b24f4fb4cca0dfa53cd969) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:32.052538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:32.053949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157227a-f2e4-47aa-a2f3-b00f0ffbcfae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:32.059120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 586/1185] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:14:31 2022] GET /identity/v3/role_assignments?user.id=fa378b7521b24f4fb4cca0dfa53cd969&scope.project.id=c2e6d91256e54c4493211d86d75e0ca2&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:34.024574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c834ab9-d158-4b5c-b945-f5ed00470535 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.025127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c834ab9-d158-4b5c-b945-f5ed00470535 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.025127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c834ab9-d158-4b5c-b945-f5ed00470535 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.026444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 600/1186] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:34.031588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.032047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.032153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.049841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:34.051032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:34.114098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 587/1187] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:14:34.136292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c834ab9-d158-4b5c-b945-f5ed00470535 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:34.201993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:34.234383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c834ab9-d158-4b5c-b945-f5ed00470535 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:34.268012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:34.268657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.268987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.269516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.278896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:34.283666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:34.284860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:14:34.285866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 601/1188] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:34.291049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:34.326795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:34.390569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edba35-4079-442d-bb88-27f2433909e4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:34.422668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:34.423521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.423797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.424148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.424869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:34.425168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:34.427027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:34.434752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 588/1189] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:34.440626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:34.536549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c853-ca7b-4f24-ae22-872339fb575a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:34.562344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:34.570250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:34.570871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.571149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.571422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] PATH_INFO: `/v3/users/fa378b7521b24f4fb4cca0dfa53cd969` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.572572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] RBAC: Authorizing `identity:get_user(user_id=fa378b7521b24f4fb4cca0dfa53cd969)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:34.573980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:34.575534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 602/1190] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:34 2022] GET /identity/v3/users/fa378b7521b24f4fb4cca0dfa53cd969 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:34.581237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:34.674865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:34.681657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59062-314f-42a5-9e93-f9a979fea4d3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:34.713224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:34.713852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.714136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.714412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] PATH_INFO: `/v3/projects/c2e6d91256e54c4493211d86d75e0ca2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.715554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e6d91256e54c4493211d86d75e0ca2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:34.717005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbb401-f0e1-44d0-adf1-11ab4b7bedf9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:34.718639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 589/1191] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:34 2022] GET /identity/v3/projects/c2e6d91256e54c4493211d86d75e0ca2 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:34.725242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:34.821671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359537c-e37d-4f72-859f-83764c224b98 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:34.865167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:34.865823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:34.866102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:34.866376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] PATH_INFO: `/v3/projects/c2e6d91256e54c4493211d86d75e0ca2/users/fa378b7521b24f4fb4cca0dfa53cd969/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:34.868302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e6d91256e54c4493211d86d75e0ca2, user_id=fa378b7521b24f4fb4cca0dfa53cd969, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:34.869725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3132b-9dd4-4c76-9f34-dfe0968a4471 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:34.881946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 603/1192] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:14:34 2022] PUT /identity/v3/projects/c2e6d91256e54c4493211d86d75e0ca2/users/fa378b7521b24f4fb4cca0dfa53cd969/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:14:36.851830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09904-4b57-40b6-b23e-5ac6550bcaf3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:36.852608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09904-4b57-40b6-b23e-5ac6550bcaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:36.852608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09904-4b57-40b6-b23e-5ac6550bcaf3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:36.854009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 590/1193] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:14:36.859714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:36.860169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:36.860311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:36.877904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:14:36.879685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:14:36.929021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:36.946649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 604/1194] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:14:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:14:36.970003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09904-4b57-40b6-b23e-5ac6550bcaf3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.067081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09904-4b57-40b6-b23e-5ac6550bcaf3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.098933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.099547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.099824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.100125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.110142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:14:37.114968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:14:37.116168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:14:37.117230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 591/1195] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:14:37.122162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.216898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260cb5-74fa-4d4a-b1da-37b151896085 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.249829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.250439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.250713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.250983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.251684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:37.252008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:37.254048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:37.261849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 605/1196] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:37.267507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.293447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:37.366014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806f07e-b15e-4f8f-bde6-8d7846d768c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.397660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.398271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.398544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.398813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] PATH_INFO: `/v3/users/fa378b7521b24f4fb4cca0dfa53cd969` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.399942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] RBAC: Authorizing `identity:get_user(user_id=fa378b7521b24f4fb4cca0dfa53cd969)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:37.401378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:37.402915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 592/1197] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Aug 4 10:14:37 2022] GET /identity/v3/users/fa378b7521b24f4fb4cca0dfa53cd969 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:37.408622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.456328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:37.505314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4357f-8e53-4eeb-a37c-a96efda79805 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.536892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.537491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.537770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.538043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.602471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:37.603910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:37.647901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:37.697570 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 606/1198] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:14:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:14:37.703455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.790146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:37.799456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e436-b78e-450b-bf5b-a0af73946257 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.832593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.833247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.833529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.833803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] PATH_INFO: `/v3/projects/c2e6d91256e54c4493211d86d75e0ca2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.834933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e6d91256e54c4493211d86d75e0ca2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:37.836379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ac02c-8865-4c6a-8b5b-0e77019668e9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:37.838024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 593/1199] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Aug 4 10:14:37 2022] GET /identity/v3/projects/c2e6d91256e54c4493211d86d75e0ca2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:37.844042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:14:37.938863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a52d-d511-4b4d-a5a8-6b03503734b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:14:37.970790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:14:37.971391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:14:37.971666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:14:37.971934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:14:37.973414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=c2e6d91256e54c4493211d86d75e0ca2, user.id=fa378b7521b24f4fb4cca0dfa53cd969) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:14:37.973726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:14:37.975138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e3e30-176e-4931-b9a7-c98bebcd9053 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:14:37.980719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 607/1200] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:14:37 2022] GET /identity/v3/role_assignments?user.id=fa378b7521b24f4fb4cca0dfa53cd969&scope.project.id=c2e6d91256e54c4493211d86d75e0ca2&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:14:47.828342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=20 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:14:47.978205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:33.044710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37cd36-3f87-4656-9189-706ab751e035 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:33.044710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37cd36-3f87-4656-9189-706ab751e035 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:33.044710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37cd36-3f87-4656-9189-706ab751e035 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:33.044710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 594/1201] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:33.049982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:33.050622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:33.050622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:33.075907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:33.077130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:33.163987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 608/1202] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:33.185801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37cd36-3f87-4656-9189-706ab751e035 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:33.331052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37cd36-3f87-4656-9189-706ab751e035 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:33.374413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:33.375117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:33.375390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:33.375736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:33.390566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:33.392179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:33.417922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:33.419052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:33.420075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:33.420075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:33.420075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:33.420075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:33.420075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 595/1203] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:33.426055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:33.562579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e154647-599d-4069-b564-a1a9798379b6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:33.609410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:33.615624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:33.616301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:33.616784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:33.617128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:33.617832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:33.618146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:33.620013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fa542-c4f9-4d09-806b-7bebb53d2e0b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:33.632543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 609/1204] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:33.639365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:33.713254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:33.777415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847036bb-0c9f-4064-8ca8-0f2bc28858b8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:33.828975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:33.829938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:33.830424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:33.830764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:33.831613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:33.833356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050868a3-ffa9-4074-bc5c-2c7f7f728117 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:33.869066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 596/1205] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Aug 4 10:15:33 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:15:36.075310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae624c-1f51-4810-a296-3e21891f6285 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.075891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae624c-1f51-4810-a296-3e21891f6285 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.075891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae624c-1f51-4810-a296-3e21891f6285 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.077498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 610/1206] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:36.083467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.083906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.084144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.103712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:36.105093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:36.187798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 597/1207] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:36.215128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae624c-1f51-4810-a296-3e21891f6285 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:36.335420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae624c-1f51-4810-a296-3e21891f6285 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:36.374306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:36.374951 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.375236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.375581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.390079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:36.392821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:36.393970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:36.395051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:36.399241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 611/1208] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:15:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:36.400230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:36.444245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:36.517836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3092-74f4-40ee-bd2f-20a3433f0803 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:36.555773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:36.556430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.556746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.557057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.557851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:36.558147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:36.559577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:36.573728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 598/1209] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:15:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:36.579423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:36.635589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:36.695906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c938d4f-c005-4fc4-8030-f7f4e9ad840b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:36.739610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:36.740277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.740559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.740831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.753364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:36.755212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:36.772303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:36.773661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:36.774763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:36.774763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:36.774763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:36.774763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:36.778599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 612/1210] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:36.779642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:36.896085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:36.899825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b0356-26de-47d7-b709-e8fe30f291bb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:36.942167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:36.942818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:36.943100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:36.943455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:36.944407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:36.944725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:36.946226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:36.963116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 599/1211] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:36.970532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:37.099078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb5eb-c161-457e-a737-c6f6cf5068f5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:37.101212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:37.141176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da618-1733-440f-9afb-6694c9682570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:37.141821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da618-1733-440f-9afb-6694c9682570 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:37.142110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da618-1733-440f-9afb-6694c9682570 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:37.142385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da618-1733-440f-9afb-6694c9682570 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:37.152018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502da618-1733-440f-9afb-6694c9682570 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:37.153812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502da618-1733-440f-9afb-6694c9682570 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-502da618-1733-440f-9afb-6694c9682570 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:37.166958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:37.168022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:37.169003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 613/1212] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:37.174371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:37.295861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ece17-31a8-4e8a-b2ae-6805b85b85ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:37.335017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:37.336030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:37.336470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:37.336831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:37.337738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:37.338222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:37.339795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:37.359471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 600/1213] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:37.365674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da618-1733-440f-9afb-6694c9682570 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:37.480005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da618-1733-440f-9afb-6694c9682570 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:37.516154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:37.516778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:37.517043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:37.517311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:37.551073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:37.594094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:37.595893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:37.722261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 614/1214] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:15:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:15:37.729232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:37.753213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:37.857334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750816-e421-4a72-a4c0-be8e46d5a37c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:37.908323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:37.909290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:37.909771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:37.910362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:37.925316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:37.927649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:37.953059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:37.954058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:37.960243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:37.960243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:37.960243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:37.960243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:37.960243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 601/1215] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:37.962105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:38.043161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:38.089527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2daec-5ad0-4a2a-ae90-c0b920f938dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:38.135806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:38.137317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:38.137799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:38.138253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:38.139194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:38.139717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:38.141225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:38.154770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 615/1216] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:38.160950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:38.253352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:38.288203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a022e6-4ae7-46be-aa0f-858853ac951c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:38.325014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:38.325580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:38.325924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:38.326138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:38.332882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:38.334324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:15:38.345532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:38.346497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:38.347437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 602/1217] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:38.353658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:38.473611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32166651-8827-48fb-aa9b-bc08e287d2c8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:38.497256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:38.511491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:38.512232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:38.512554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:38.512928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:38.513687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:38.513988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:38.515484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1f3-c92e-4028-b265-a1894658fb44 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:38.536880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 616/1218] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:38.538841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:38.660358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5416f-09a3-4435-8b8d-b85f4b49ed89 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:38.691245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:38.698187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:38.698799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:38.699077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:38.699350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:38.700217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=47da679abbac42a1b3244ff9acafd577) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:38.700529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:38.701950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997391d8-b81c-436e-98c3-f1625cc46ebd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:38.708094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 603/1219] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:15:38 2022] GET /identity/v3/role_assignments?user.id=47da679abbac42a1b3244ff9acafd577&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:40.825251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacf1fb-03ee-4c77-bc92-21ec15b2ff66 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:40.825774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacf1fb-03ee-4c77-bc92-21ec15b2ff66 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:40.825774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacf1fb-03ee-4c77-bc92-21ec15b2ff66 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:40.828180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 617/1220] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:40.834396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:40.834935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:40.835062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:40.855824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:40.857269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:40.938391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 604/1221] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:40.963857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacf1fb-03ee-4c77-bc92-21ec15b2ff66 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.083124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacf1fb-03ee-4c77-bc92-21ec15b2ff66 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:41.123019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:41.123867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:41.124268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:41.124671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:41.136964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:41.143013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:41.144277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:41.145832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 618/1222] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:41.151390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.263144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671221c-e817-4f76-99fa-b1b5a1461de9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:41.292671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:41.305207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:41.305826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:41.306105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:41.306376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:41.307085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:41.307389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:41.308787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:41.316727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 605/1223] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:41.323303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.377400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:41.437388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ef4-0d43-4e51-931a-631af032db31 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:41.472564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:41.473721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:41.474003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:41.474266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:41.484537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:41.486008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:41.501133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:41.502285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:41.503261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:41.503261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:41.503261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:41.503261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:41.503261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 619/1224] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:41.508461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.632182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83140f79-5f7e-4676-9f52-b2b82c154911 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:41.668257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:41.668880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:41.669164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:41.669440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:41.670127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:41.670433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:41.671794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:41.684420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 606/1225] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:41.690026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.789940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:41.796809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaf88d-83be-4a7d-a7e5-944b175b23dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:41.830505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:41.831265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:41.831560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:41.831820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:41.840153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:41.841703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:41.853355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:41.854424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:41.855357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 620/1226] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:41.860311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:41.911479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:41.984460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33436a5f-35f8-4a57-b047-cca15354bef9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.028928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.029673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.030161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.030512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.031528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:42.031949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.034037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.051370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 607/1227] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:42.057032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:42.163421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff8941-81b6-46c0-9565-6dff29446e5a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.200263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.201034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.201318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.201723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.213800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.215453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:42.232067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:42.233171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:42.234399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:42.234399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:42.234399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:42.234399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:42.234399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 621/1228] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:42.239991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:42.354743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971749d-20a5-485c-8ff8-c068a5895ebe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.357069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:42.392764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.393474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.393757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.394023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.394768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:42.395070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.396643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.409311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 608/1229] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:42.416004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:42.425847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:42.532730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e328-3450-4d8b-8292-916172197d84 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.573361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.574123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.574428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.574702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.582744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.584767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:15:42.598071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:42.599568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:42.600681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 622/1230] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:42.605272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:42.718142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fce2b-fca4-4466-8e9e-808ea56313dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.754985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac4782-f402-429a-94d2-62719840946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.755601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4782-f402-429a-94d2-62719840946d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.755876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4782-f402-429a-94d2-62719840946d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.756201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4782-f402-429a-94d2-62719840946d None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.756958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4782-f402-429a-94d2-62719840946d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:42.757250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4782-f402-429a-94d2-62719840946d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.758643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4782-f402-429a-94d2-62719840946d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.773304 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 609/1231] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:42.779651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:42.856566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:42.897546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6a39-63dd-4a7e-b8a7-b4cdd267823c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:42.934991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:42.935655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:42.935934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:42.936231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/47da679abbac42a1b3244ff9acafd577/roles/61d97d6f49bf4d6d818b7312a856d5bd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:42.953245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=47da679abbac42a1b3244ff9acafd577, role_id=61d97d6f49bf4d6d818b7312a856d5bd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:42.955029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fe00a-682a-424d-ad57-0137fcf376a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:42.970412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 623/1232] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:15:42 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/47da679abbac42a1b3244ff9acafd577/roles/61d97d6f49bf4d6d818b7312a856d5bd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:15:45.326167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50dc4-ed1f-4fc3-ac70-d51fa4c98468 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:45.326712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50dc4-ed1f-4fc3-ac70-d51fa4c98468 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:45.326712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50dc4-ed1f-4fc3-ac70-d51fa4c98468 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:45.328211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 610/1233] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:45.337541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:45.337998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:45.338089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:45.356728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:45.357952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:45.427558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 624/1234] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:45.451374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50dc4-ed1f-4fc3-ac70-d51fa4c98468 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:45.476731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:45.557593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50dc4-ed1f-4fc3-ac70-d51fa4c98468 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:45.592707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:45.593352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:45.593634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:45.593910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:45.605575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:45.611890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:45.612896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:45.619623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 611/1235] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Aug 4 10:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:45.620839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:45.667977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:45.727816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3516e2-036a-472b-97e1-c82da6a67f29 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:45.761481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:45.762095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:45.762375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:45.762648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:45.763359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:45.763663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:45.765068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:45.773929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 625/1236] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Aug 4 10:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:45.779764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:45.872763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:45.889299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552afd8-0f60-4a38-a51b-1522a8b4aa2a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:45.925798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:45.927010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:45.927285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:45.927560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:45.937226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:45.938652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:45.954182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:45.955328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:45.956383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:45.956383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:45.956383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:45.956383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:45.956383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 612/1237] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:45.961859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:46.081933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861fa1-38ce-46ff-9f00-13052ea3ea15 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:46.091709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:46.120484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:46.121183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:46.121412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:46.121718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:46.122505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:46.122861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:46.124392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:46.139984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 626/1238] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:46.143496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:46.266508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f4131-9ab9-4174-980a-0e01300f6c33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:46.303396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:46.304066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:46.304350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:46.304637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:46.307491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:46.313408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:46.315273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:46.327938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:46.329475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:46.330427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 613/1239] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:46.336526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:46.447503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d43962-e990-46e3-9bf1-fbc8caa73bce None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:46.485581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:46.486254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:46.486563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:46.486861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:46.487775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:46.488116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:46.489760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:46.509557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 627/1240] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:46.515430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:46.630692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b039f-55fe-488d-bec2-c02f0a0648db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:46.668074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:46.668938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:46.669216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:46.669588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:46.741283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:46.749393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:46.751015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:46.866672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 614/1241] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:15:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:15:46.872588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:46.902606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:46.987172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0466-d8f4-4d57-9156-0d253a820232 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:47.028557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:47.029253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:47.029540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:47.029855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:47.042662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:47.044297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:47.062506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:47.063631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:47.064679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:47.064679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:47.064679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:47.064679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:47.064679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 628/1242] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:47.069670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:47.171387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:47.178716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7cfc3-0044-49d7-8bd8-06aa65fe95ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:47.220177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:47.220796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:47.221081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:47.221421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:47.222203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:47.222515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:47.224681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:47.237191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 615/1243] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:47.243778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:47.327993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:47.352556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196692d4-22cb-4b14-aa76-7eadd341a756 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:47.408836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:47.409509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:47.409819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:47.410165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:47.419524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:47.419944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:15:47.434447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:47.435456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:47.436991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 629/1244] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:47.442271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:47.562669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156515a1-49e4-46d3-9747-ef72e1b9c950 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:47.602354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:47.602901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:47.603254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:47.603577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:47.604408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:47.604782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:47.606531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8489e-f345-44dc-8e40-1c09691ee527 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:47.621957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 616/1245] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:47.629877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:47.738161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52401780-c5d4-43fc-b6a6-6ad8b692f661 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:47.775914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ced539-71c6-439f-b116-df263742179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:47.776553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced539-71c6-439f-b116-df263742179e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:47.776829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced539-71c6-439f-b116-df263742179e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:47.777098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced539-71c6-439f-b116-df263742179e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:47.777875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced539-71c6-439f-b116-df263742179e None admin] RBAC: Adding query filter params (role.id=61d97d6f49bf4d6d818b7312a856d5bd, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=47da679abbac42a1b3244ff9acafd577) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:47.778177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced539-71c6-439f-b116-df263742179e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:47.779748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced539-71c6-439f-b116-df263742179e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:47.786334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 630/1246] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:15:47 2022] GET /identity/v3/role_assignments?user.id=47da679abbac42a1b3244ff9acafd577&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=61d97d6f49bf4d6d818b7312a856d5bd => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:50.094581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dcc92-45e2-4ecc-ae13-b7c0befeed4b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.095144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dcc92-45e2-4ecc-ae13-b7c0befeed4b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.095241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dcc92-45e2-4ecc-ae13-b7c0befeed4b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.096736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 617/1247] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:50.103524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.105544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.105544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.110709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:50.124776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:50.126464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:50.202778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 631/1248] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:50.229189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058dcc92-45e2-4ecc-ae13-b7c0befeed4b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:50.252522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:50.352112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058dcc92-45e2-4ecc-ae13-b7c0befeed4b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:50.392531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:50.393168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.393462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.393727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.405194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:50.410568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:50.411771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:50.412849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 618/1249] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:50.418345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:50.533196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cbb81-e83d-40a2-b10c-5cba926f0a8f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:50.571817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:50.578173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:50.578831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.579087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.579390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.580158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:50.580461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:50.581841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:50.591286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 632/1250] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:50.597590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:50.683705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:50.709251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da9e1-1b1a-44a4-8e39-508524782ebc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:50.748493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:50.749175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.749551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.749809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.761869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:50.763363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:50.778846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:50.780073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:50.780960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:50.780960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:50.780960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:50.780960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:50.780960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 619/1251] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:50.786167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:50.894704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22cc4-ba77-46de-bebe-39b314d7a9d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:50.933880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:50.934407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:50.934687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:50.934959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:50.935654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:50.935979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:50.937341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:50.950508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 633/1252] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:50.956427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:51.084786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62b5c-aa1f-41a0-b015-bb45150f01d3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:51.118207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:51.126735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:51.127619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:51.127947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:51.128267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:51.136165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:51.137689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:51.149575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:51.151387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:51.153365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 620/1253] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:51.156693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:51.228087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:51.278762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c06d-e30b-4c65-9e0d-4a7382b7b481 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:51.315901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:51.316625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:51.316906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:51.317182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:51.317969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:51.318273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:51.319699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:51.337070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 634/1254] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:51.343148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:51.451436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af335dc-29b5-4fda-bd15-81c3709c2d24 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:51.490670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:51.491123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:51.491401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:51.491678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:51.558962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:51.572334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:51.573827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:51.683527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 621/1255] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:15:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:15:51.689399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:51.811064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa32a-b6bf-4623-80c0-c3f38ce39447 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:51.828925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:51.854129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:51.854769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:51.855046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:51.855315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:51.865157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:51.866590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:51.881435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:51.882476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:51.883382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:51.883382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:51.883382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:51.883382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:51.883382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 635/1256] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:51.888449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:51.982855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:51.995345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42cee-852e-4dac-b917-8f7e7859a85a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:52.039841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:52.040797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:52.041115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:52.041386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:52.042355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:52.042690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:52.044854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:52.058177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 622/1257] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:52.064737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:52.190064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2586e-f16e-4a4d-976b-e36d9ffa7a9d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:52.230001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:52.230646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:52.230926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:52.231201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:52.238904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:52.240403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:15:52.256905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:52.258085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:52.259234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 636/1258] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:15:52.264721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:52.370676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84595f40-719a-4d96-9bf5-bfd61eb79866 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:52.404993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:52.405610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:52.405893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:52.406168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:52.406910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:52.407218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:52.408676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5edc5-9ad9-41bb-bea2-40f9fef40ab3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:52.421455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 623/1259] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:52.428246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:52.458163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:52.544134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c4792-2f2d-4002-a514-c405301d9e90 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:52.580127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:52.580780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:52.581108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:52.581405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:52.582864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=47da679abbac42a1b3244ff9acafd577) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:52.583178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:52.584680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1fa3a-6bd6-4e2d-a97c-bea246b07a5c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:52.590201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 637/1260] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:15:52 2022] GET /identity/v3/role_assignments?user.id=47da679abbac42a1b3244ff9acafd577&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:54.583878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a00c6-064d-4a5f-b422-e2f2c93ae206 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:54.584430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a00c6-064d-4a5f-b422-e2f2c93ae206 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:54.584524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a00c6-064d-4a5f-b422-e2f2c93ae206 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:54.585820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 624/1261] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:54.590882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:54.591317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:54.591435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:54.609573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:54.610743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:54.674643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 638/1262] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:54.696374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a00c6-064d-4a5f-b422-e2f2c93ae206 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:54.707616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:54.794842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a00c6-064d-4a5f-b422-e2f2c93ae206 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:54.827101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:54.827719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:54.828026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:54.828298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:54.838561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:54.843336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:54.844463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:54.845350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 625/1263] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:54.850318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:54.867646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:54.943938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d31560-36c2-4cf6-be07-8bb01a5e7311 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:54.976126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:54.976740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:54.977019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:54.977292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:54.978007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:54.978310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:54.979711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:54.987540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 639/1264] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:15:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:54.993963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.070816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:55.103088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b79d47-38e3-42a8-b4a2-57bf25cee3c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.136194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.136818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.137096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.137366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.146931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.148418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:55.163259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:55.164302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:55.165236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:55.165236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:55.165236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:55.165236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:55.165236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 626/1265] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:55.170261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.248469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:55.267541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e18fd8-723c-418c-9fcf-5851d17639f0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.299228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.299828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.300410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.300685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.301889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:55.302197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.303553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.314873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 640/1266] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:55.321200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.415723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84796a-4522-4d1a-bed5-91e30327439a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.435071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:55.449393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.450012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.450290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.450560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.457841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.459282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:55.470016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:55.471127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:55.472308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 627/1267] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:55.477459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.573313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f413ed-72dd-4493-86e1-84f232af502e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.605312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.605909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.606173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.606441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.607217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:55.607517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.608945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.617768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:55.626631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 641/1268] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:55.632202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.727505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116efb6-7fb3-48c4-8f09-ecc5d7d6ae39 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.759389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.760030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.760315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.760588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.770666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.772114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:55.786200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:55.787267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:55.788423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:55.788423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:55.788423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:55.788423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:55.788423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 628/1269] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:55.792972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.899539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabc28-9e99-40eb-9aa0-50c154feaf26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:55.943146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:55.943748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:55.944061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:55.944334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:55.945011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:55.945310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:55.946652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:55.957947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 642/1270] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:55.963698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:55.995661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:56.074922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb23b-18d4-4bb6-b771-1378452bff3c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:56.107762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:56.108418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:56.108701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:56.108974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:56.115441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:56.116896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:15:56.127847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:56.128922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:15:56.129993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:56.132104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 629/1271] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:56.134899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:56.191184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:56.231810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f160b03-e66a-49e5-8ec4-6b88d22e5b52 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:56.265202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:56.265816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:56.266097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:56.266370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:56.267117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:56.267425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:56.268881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616e5d2-ee0c-4547-827d-f4f6dc1ecda0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:56.281296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 643/1272] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:56.287483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:56.375218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:56.384453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5c885-659c-46f8-987f-d23f8be06c00 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:56.417317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:56.417966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:56.418250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:56.418526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] PATH_INFO: `/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/47da679abbac42a1b3244ff9acafd577/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:56.420449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] RBAC: Authorizing `identity:create_grant(project_id=212cd92bbff94ea9af0e07b03238b9ad, user_id=47da679abbac42a1b3244ff9acafd577, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:56.421868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2929a7b-176b-405e-bc54-2967f3e7de91 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:56.434808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 630/1273] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Aug 4 10:15:56 2022] PUT /identity/v3/projects/212cd92bbff94ea9af0e07b03238b9ad/users/47da679abbac42a1b3244ff9acafd577/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:15:58.404500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5cef5-9573-4c7a-a5db-abcdbfc47b44 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:58.405017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5cef5-9573-4c7a-a5db-abcdbfc47b44 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:58.405017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5cef5-9573-4c7a-a5db-abcdbfc47b44 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:58.406407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 644/1274] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:15:58.412182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:58.412563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:58.412742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:58.431249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:15:58.432467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:15:58.495747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 631/1275] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:15:58.519352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5cef5-9573-4c7a-a5db-abcdbfc47b44 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:58.618620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5cef5-9573-4c7a-a5db-abcdbfc47b44 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:58.634941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:58.651477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:58.652142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:58.652421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:58.652688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:58.662845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:15:58.667632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:58.668771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:58.669989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 645/1276] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Aug 4 10:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:58.674842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:58.752538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:58.770973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e635e-1db2-4528-8e2a-25d597a7e62f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:58.804067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:58.804668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:58.804945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:58.805215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:58.805916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:58.806217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:58.807584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:58.815249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 632/1277] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Aug 4 10:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:58.820939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:58.917282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691b556-3fe0-4564-a15b-d9e5fd2f4b47 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:58.949198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:58.949822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:58.950100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:58.950369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:58.960429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:58.961852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:58.976465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:58.977468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:58.978543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:58.978543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:58.978543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:58.978543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:58.978543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 646/1278] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:58.983171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:59.087946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23506a-09d0-44ab-af59-e23c0d8a15c4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:59.120205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:59.120808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:59.121089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:59.121362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:59.122039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:59.122343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:59.123687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:59.134891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 633/1279] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:15:59.140606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:59.171738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:59.238540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882963-31d4-40b9-bd8e-aaae3e6fdc83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:59.271502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:59.272160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:59.272438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:59.272706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:59.279760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:59.281214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:15:59.292561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:59.293652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:15:59.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 647/1280] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:59.299377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:59.303938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:59.396066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca763-5aed-4be7-acc3-980a523ad1b1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:59.428321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:59.428920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:59.429201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:59.429473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:59.430256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:15:59.430561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:59.431992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:59.448623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 634/1281] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Aug 4 10:15:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:15:59.454300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:59.542384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:59.552547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9cc2b-130a-4c5e-9061-5b2ee22ffb2d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:59.586414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:59.587036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:59.587316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:59.587586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:59.652085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:59.653468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:59.737445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:59.748140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 648/1282] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Aug 4 10:15:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:15:59.754030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:15:59.813498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:15:59.851165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a56-c64b-4d10-8447-f0d458ca1eaf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:15:59.884496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:15:59.885140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:15:59.885420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:15:59.885693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:15:59.896978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:15:59.898437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:15:59.914564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:15:59.915674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:15:59.916961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:15:59.916961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:15:59.916961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:15:59.916961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:15:59.916961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 635/1283] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:15:59.921567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:00.029848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6d31-c91a-4d4a-94c0-5bf6ede45903 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:00.064849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:00.065463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:00.065745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:00.066017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:00.067212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:00.067513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:00.068910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:00.079651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 649/1284] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Aug 4 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:00.086173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:00.181969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb134d-12a0-4398-a515-91095a84af47 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:00.195212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:00.217608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:00.218235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:00.218506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:00.218774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:00.225343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:00.226765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:16:00.237614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:16:00.238724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 04 10:16:00.239687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 636/1285] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Aug 4 10:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:16:00.245049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:00.294707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:00.343600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20eb7-3ed8-46ac-a63d-11cf5d2901e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:00.376021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:00.376634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:00.376913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:00.377187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:00.377921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:00.378225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:00.379667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492b31d-8977-46d1-b17c-c6d4ccbe809c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:00.391615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 650/1286] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Aug 4 10:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:00.398245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:00.495042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa2c3-e5b3-456a-b395-b03c7ec652c0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:00.527184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:00.527773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:00.528088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:00.528379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:00.529175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] RBAC: Adding query filter params (role.id=abf8a86280c54a0798604e6c88e690cf, scope.project.id=212cd92bbff94ea9af0e07b03238b9ad, user.id=47da679abbac42a1b3244ff9acafd577) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:00.529486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:00.530928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425d971-1fd0-41b7-adec-5d23768bb9b8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:00.537058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 637/1287] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Aug 4 10:16:00 2022] GET /identity/v3/role_assignments?user.id=47da679abbac42a1b3244ff9acafd577&scope.project.id=212cd92bbff94ea9af0e07b03238b9ad&role.id=abf8a86280c54a0798604e6c88e690cf => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:02.522184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85699a43-52c9-48e3-ac52-58c621e508f7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:02.522752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85699a43-52c9-48e3-ac52-58c621e508f7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:02.522752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85699a43-52c9-48e3-ac52-58c621e508f7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:02.524098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 651/1288] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:16:02.529130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:02.529515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:02.529673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:02.547734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:16:02.548961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:16:02.578665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:02.616083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 638/1289] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:16:02.643784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85699a43-52c9-48e3-ac52-58c621e508f7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:02.741450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85699a43-52c9-48e3-ac52-58c621e508f7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:02.751497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:02.774376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:02.774942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:02.775219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:02.775489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:02.776138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:02.777491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:16:02.786625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:16:02.787857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:02.788828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 652/1290] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:16:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:16:02.793769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:02.888895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ce235-2733-406d-8621-d87e4fa98254 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:02.921935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:02.922550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:02.922831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:02.923105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:02.923863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:02.924194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:02.925867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79418425-4497-4438-ace5-c622ce87bd5e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:02.930673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 639/1291] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Aug 4 10:16:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:02.936508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:03.052004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37d587-1d75-4e34-91d7-dd36b7040740 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:03.085667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:03.086280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:03.086559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:03.086832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:03.087496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:03.087796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:03.089182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e58ce-dbff-4866-8361-782c56dc720e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:03.093435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 653/1292] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Aug 4 10:16:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:05.057562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34689b-aa63-46b2-a237-a8d93c00f2fb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:05.058120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34689b-aa63-46b2-a237-a8d93c00f2fb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:05.058120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34689b-aa63-46b2-a237-a8d93c00f2fb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:05.059443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 640/1293] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:16:05.064727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396855f-0150-4fa1-846e-f3dc021f67a5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:05.065122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396855f-0150-4fa1-846e-f3dc021f67a5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:05.065277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396855f-0150-4fa1-846e-f3dc021f67a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:05.083440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a396855f-0150-4fa1-846e-f3dc021f67a5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:16:05.084633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a396855f-0150-4fa1-846e-f3dc021f67a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:16:05.112508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:05.149370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 654/1294] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:16:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:16:05.171048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34689b-aa63-46b2-a237-a8d93c00f2fb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:05.202038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:05.270858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34689b-aa63-46b2-a237-a8d93c00f2fb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:05.303839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:05.304462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:05.304742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:05.305015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:05.305708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:05.307051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ff117-3cdf-4e52-bdb1-602a4d6f948b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:05.320814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 641/1295] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Aug 4 10:16:05 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:16:07.296850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed064a-4e39-4cc9-9b4a-3885f7d99b6c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.297348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed064a-4e39-4cc9-9b4a-3885f7d99b6c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.297465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed064a-4e39-4cc9-9b4a-3885f7d99b6c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.298756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 655/1296] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:16:07.304517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.304907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.305066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.323821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:16:07.325022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:16:07.394210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 642/1297] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:16:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:16:07.416090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed064a-4e39-4cc9-9b4a-3885f7d99b6c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:07.513845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed064a-4e39-4cc9-9b4a-3885f7d99b6c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:07.546009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:07.546635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.546915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.547186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.547802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:07.549185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:16:07.558639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:16:07.559632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:07.560657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 656/1298] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:16:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:16:07.565846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:07.584741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:07.663520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2b5c6-4f65-4fea-b8ec-e7cc54aeaebb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:07.696234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:07.696837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.697118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.697394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.698066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:07.698370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:07.699724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:07.704733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 643/1299] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Aug 4 10:16:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:07.711303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:07.741504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:07.807662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d344db-d841-4ca4-bf39-2b355a45c424 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:07.840014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:07.840615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.840895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.841169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.841916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] RBAC: Adding query filter params (interface=public, service_id=cc213217c4ea46d6a99f1828621110d4, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:07.842222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:07.843574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec069915-a632-447a-9406-31dd51e75eea None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:07.848435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 657/1300] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Aug 4 10:16:07 2022] GET /identity/v3/endpoints?service_id=cc213217c4ea46d6a99f1828621110d4&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:07.853938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:07.932841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:07.950716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72292-fdb1-4410-9a63-7f6cfa76d2a7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:07.985138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:07.985753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:07.986038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:07.986306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:07.986949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:07.988330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737adec-47e5-4e36-8944-c546b788e5f0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:07.994981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 644/1301] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Aug 4 10:16:07 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:16:10.001476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa54-71b7-47a5-850a-d953f2afb5ca None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:10.002027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa54-71b7-47a5-850a-d953f2afb5ca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:10.002148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa54-71b7-47a5-850a-d953f2afb5ca None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:10.004209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 658/1302] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:16:10.010936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:10.011406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:10.011556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:10.034463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:16:10.036531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:16:10.111501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 645/1303] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:16:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:16:10.134301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fa54-71b7-47a5-850a-d953f2afb5ca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:10.226673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:10.233743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fa54-71b7-47a5-850a-d953f2afb5ca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:10.265710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:10.266339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:10.266618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:10.266889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:10.267565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:10.268958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:16:10.278081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:16:10.279202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Aug 04 10:16:10.280061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 659/1304] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Aug 4 10:16:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:16:10.287177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:10.359744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:10.383260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3926-406f-4937-abf2-ff3ed79576f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:10.415377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:10.415996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:10.416274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:10.416542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:10.417201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:16:10.417497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:10.418834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3b957-96be-4f7d-91be-ed661a2fff3f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:10.423833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 646/1305] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Aug 4 10:16:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:16:10.430347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:16:10.525867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9e87-5c39-4c18-a8d4-4b92facf2be7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:16:10.557524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:16:10.558115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:16:10.558389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:16:10.558657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:16:10.559352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:16:10.560706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce34902-8ff9-4efd-873a-2b38d3853cff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:16:10.580220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 660/1306] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:16:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:16:20.420895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=30 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:16:20.575896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:13.522121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44014-8d38-44ba-af40-1c15f4db404d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:13.523189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44014-8d38-44ba-af40-1c15f4db404d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:13.523692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44014-8d38-44ba-af40-1c15f4db404d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:13.529272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 647/1307] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:13.530662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2426-9fa3-4538-8071-3aa1db3bf26f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:13.531102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2426-9fa3-4538-8071-3aa1db3bf26f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:13.531321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2426-9fa3-4538-8071-3aa1db3bf26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:13.609374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e49f2426-9fa3-4538-8071-3aa1db3bf26f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:13.611049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e49f2426-9fa3-4538-8071-3aa1db3bf26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:13.740303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 661/1308] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:17:13.810438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:13.810731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:13.810939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:13.812564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 648/1309] 10.222.0.72 () {54 vars in 965 bytes} [Thu Aug 4 10:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:13.821323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee7b5-a9c9-436e-8f3d-52a36e8c1975 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:13.822022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee7b5-a9c9-436e-8f3d-52a36e8c1975 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:13.822577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee7b5-a9c9-436e-8f3d-52a36e8c1975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:13.971299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a0aee7b5-a9c9-436e-8f3d-52a36e8c1975 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:13.974453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a0aee7b5-a9c9-436e-8f3d-52a36e8c1975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:14.002836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:14.070140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 662/1310] 10.222.0.72 () {58 vars in 1062 bytes} [Thu Aug 4 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:17:14.079222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:14.081112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:14.203886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d383c0-1db4-403f-8521-7083c56a87be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:14.294202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:14.294737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:14.295014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:14.295286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:14.296930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:14.351941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:14.409826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:14.411757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f4f3d-3b1b-47f6-bc87-4f50a6da5907 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:14.433089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 649/1311] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:16.756478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:16.757037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:16.757037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:16.758383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 663/1312] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:16.763609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbfb7e-20aa-4543-b20a-b0b03907daf9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:16.764056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbfb7e-20aa-4543-b20a-b0b03907daf9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:16.764195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbfb7e-20aa-4543-b20a-b0b03907daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:16.855266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ffbfb7e-20aa-4543-b20a-b0b03907daf9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:16.856538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ffbfb7e-20aa-4543-b20a-b0b03907daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:16.960708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 650/1313] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:16.998428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:16.999940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:17.110464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cba2c3-c1aa-4f96-aec7-1e00efaa49a5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:17.187877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:17.191223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:17.191869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:17.192086 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:17.192407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:17.194483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:17.315502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:17.317495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c77cc0-67f9-4ecb-92e8-1118a2ee90c1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:17.336175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 664/1314] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:19.648474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:19.649008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:19.649008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:19.650358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 651/1315] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:19.655491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34a62-066c-451a-a087-47b542f38cf5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:19.655886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34a62-066c-451a-a087-47b542f38cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:19.656064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34a62-066c-451a-a087-47b542f38cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:19.720995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-25c34a62-066c-451a-a087-47b542f38cf5 None None] MFA Rules not processed for user `a701fdc63c4941aab5144f692c2e4010`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:19.722184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25c34a62-066c-451a-a087-47b542f38cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:19.772435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:19.816515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 665/1316] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:19.855855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:19.857416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:19.867671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:19.960882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec8050-584a-4a41-9892-fe46a45ebb25 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:20.033531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:20.034139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:20.034421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:20.034695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:20.036376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:20.096807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:20.143278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:20.144748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64e188-026b-4d42-ad84-3eddf52adf0d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:20.161980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 652/1317] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:22.720377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12906ca-ff00-4b60-8b01-a249d014a1ee None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:22.720955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12906ca-ff00-4b60-8b01-a249d014a1ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:22.720955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12906ca-ff00-4b60-8b01-a249d014a1ee None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:22.722265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 666/1318] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:22.729249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d8f4f-0beb-4baa-8b06-50eb0c2ac0ea None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:22.729708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d8f4f-0beb-4baa-8b06-50eb0c2ac0ea None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:22.729805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d8f4f-0beb-4baa-8b06-50eb0c2ac0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:22.796405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8b2d8f4f-0beb-4baa-8b06-50eb0c2ac0ea None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:22.797624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8b2d8f4f-0beb-4baa-8b06-50eb0c2ac0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:22.893022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 653/1319] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:22.947525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:22.947807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:22.948116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:22.949475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 667/1320] 10.222.0.72 () {54 vars in 965 bytes} [Thu Aug 4 10:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:22.955210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eff4b-8449-4bbb-a020-28ede9f05739 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:22.955498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eff4b-8449-4bbb-a020-28ede9f05739 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:22.955773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eff4b-8449-4bbb-a020-28ede9f05739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:22.977083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:23.060432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3f7eff4b-8449-4bbb-a020-28ede9f05739 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:23.061702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f7eff4b-8449-4bbb-a020-28ede9f05739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:23.139177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 654/1321] 10.222.0.72 () {58 vars in 1062 bytes} [Thu Aug 4 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:23.148389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:23.149938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:23.253312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d0fcd-7f69-4c22-bde1-a3229ff4d067 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:23.325537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:23.326150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:23.326431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:23.326704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:23.328382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:23.335681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:23.457474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:23.458914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef8ce4-34bc-42dc-9d65-a77b480648d2 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:23.476875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 668/1322] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:26.625834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:26.626527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:26.626527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:26.627887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 655/1323] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:26.632893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee4d7-2608-43eb-af79-281c24fd1c22 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:26.633322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee4d7-2608-43eb-af79-281c24fd1c22 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:26.633440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee4d7-2608-43eb-af79-281c24fd1c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:26.699598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9beee4d7-2608-43eb-af79-281c24fd1c22 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:26.700807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9beee4d7-2608-43eb-af79-281c24fd1c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:26.764435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:26.808530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 669/1324] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:26.845257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:26.846916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:26.952600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c628e-0c1b-4719-84cf-e2a0293ed1ff None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:26.965025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:27.026556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:27.027164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:27.027437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:27.027703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:27.029320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:27.136503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:27.137820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3bc30-3031-4889-9686-3569d2864bce service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:27.155857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 656/1325] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:29.248203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:29.248627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:29.248627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:29.249894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 670/1326] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:29.255662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fde2-3a87-41b9-8d3f-f880c9984f76 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:29.256133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fde2-3a87-41b9-8d3f-f880c9984f76 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:29.256239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fde2-3a87-41b9-8d3f-f880c9984f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:29.304266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:29.322402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dcc1fde2-3a87-41b9-8d3f-f880c9984f76 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:29.323624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dcc1fde2-3a87-41b9-8d3f-f880c9984f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:29.417513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 657/1327] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:17:29.453461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:29.455076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:29.560158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b193cf-c7c3-4f52-8eb9-099f340a8270 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:29.637675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:29.638290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:29.638562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:29.638829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:29.640511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:29.658888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:29.751714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:29.753179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810529-3f81-4775-a8bc-9cae00032350 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:29.771504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 671/1328] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:31.862484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:31.863069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:31.863704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:31.865056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 658/1329] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:31.870232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7952-6212-455d-bc96-a5c7efe1ddd5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:31.870633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7952-6212-455d-bc96-a5c7efe1ddd5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:31.870779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7952-6212-455d-bc96-a5c7efe1ddd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:31.964057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-affa7952-6212-455d-bc96-a5c7efe1ddd5 None None] MFA Rules not processed for user `a701fdc63c4941aab5144f692c2e4010`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:31.966874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-affa7952-6212-455d-bc96-a5c7efe1ddd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:32.032900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:32.070592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 672/1330] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:32.108403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:32.110058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:32.213176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad20b3-18d5-40dd-8ace-cdf14fe75421 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:32.233300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:32.291380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:32.292133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:32.292445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:32.292756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:32.294796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:32.405381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:32.406829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2630470-e693-4241-b836-bf27c0052f14 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:32.426428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 659/1331] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:34.700475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:34.701096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:34.701096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:34.702386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 673/1332] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:34.707710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf60f-0b06-472b-8c9a-ec5f5b4fd721 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:34.708168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf60f-0b06-472b-8c9a-ec5f5b4fd721 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:34.708328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf60f-0b06-472b-8c9a-ec5f5b4fd721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:34.770646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:34.773462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-872cf60f-0b06-472b-8c9a-ec5f5b4fd721 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:34.775251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-872cf60f-0b06-472b-8c9a-ec5f5b4fd721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:34.869620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 660/1333] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:34.905198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:34.906756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:35.012205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32d0e-2644-45e6-acc1-8a0a9e13d409 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:35.083705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:35.084344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:35.084628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:35.084904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:35.086516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:35.112997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:35.194228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:35.195809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713488fe-66f4-4229-8af4-6bb508f03a41 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:35.214592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 674/1334] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:37.275760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:37.276306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:37.276306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:37.277655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 661/1335] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:37.282774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cf78-d17d-4e50-8d8d-1a478db4608c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:37.283200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cf78-d17d-4e50-8d8d-1a478db4608c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:37.283316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cf78-d17d-4e50-8d8d-1a478db4608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:37.348451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e854cf78-d17d-4e50-8d8d-1a478db4608c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:37.349648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e854cf78-d17d-4e50-8d8d-1a478db4608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:37.421937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:37.443804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 675/1336] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:37.484822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:37.486334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:37.588580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7597-82ad-4069-9039-f1a155a13b92 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:37.616763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:37.661533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:37.662143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:37.662427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:37.662697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:37.664362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:37.770431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:37.771856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193a008-2de2-4256-99d0-2c17bb4270ab service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:37.789411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 662/1337] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:40.194464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:40.195019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:40.195139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:40.196467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 676/1338] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:40.201554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac15bf-3826-4d6d-9ed9-124cf22b761c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:40.201970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac15bf-3826-4d6d-9ed9-124cf22b761c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:40.202101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac15bf-3826-4d6d-9ed9-124cf22b761c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:40.266466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b4ac15bf-3826-4d6d-9ed9-124cf22b761c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:40.267655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b4ac15bf-3826-4d6d-9ed9-124cf22b761c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:40.275570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:40.362117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 663/1339] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:40.399014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:40.400562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:40.501409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387a87-9b17-4195-a632-2acda22b1ee1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:40.575922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:40.576556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:40.576829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:40.577091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:40.578674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:40.613610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:40.685051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:40.686466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723ca35-1a24-4bcf-910a-17723d52c468 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:40.705705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 677/1340] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:44.624468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:44.624995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:44.625118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:44.626472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 664/1341] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:44.632440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e3b0-8e91-4a1e-9979-e6a8a1181712 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:44.632850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e3b0-8e91-4a1e-9979-e6a8a1181712 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:44.632995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e3b0-8e91-4a1e-9979-e6a8a1181712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:44.698437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0cf0e3b0-8e91-4a1e-9979-e6a8a1181712 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:44.699637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0cf0e3b0-8e91-4a1e-9979-e6a8a1181712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:44.784306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:44.796397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 678/1342] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:44.835872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:44.837465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:44.940180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f529-fe43-4bb4-b713-d3343a75c13e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:44.978442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:45.016891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:45.017512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:45.017789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:45.018080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:45.019727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:45.131736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:45.133129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a971c5-4783-4726-afa3-d4efb7cf641c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:45.150821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 665/1343] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:47.947385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:47.948088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:47.948088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:47.949428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 679/1344] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:47.954894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be012e-552c-4ef7-bd93-12f824c1e3d0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:47.955342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be012e-552c-4ef7-bd93-12f824c1e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:47.955451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be012e-552c-4ef7-bd93-12f824c1e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:48.026903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-50be012e-552c-4ef7-bd93-12f824c1e3d0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:48.028142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50be012e-552c-4ef7-bd93-12f824c1e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:48.046216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:48.126056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 666/1345] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:48.164104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:48.165640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:48.272281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944f33-6ad3-4ee9-bf9e-800d13dff152 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:48.344136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:48.344749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:48.345070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:48.345383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:48.346988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:48.389264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:48.455146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:48.456655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb342b54-e40d-4eec-9b4f-47804d5b1f45 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:48.474691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 680/1346] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:17:52.667045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:52.667570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:52.667570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:52.669001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 667/1347] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:52.674512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cff4-0e4e-49d2-b503-41cd50ad3cd3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:52.674924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cff4-0e4e-49d2-b503-41cd50ad3cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:52.675109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cff4-0e4e-49d2-b503-41cd50ad3cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:52.759257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2a14cff4-0e4e-49d2-b503-41cd50ad3cd3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:52.760912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a14cff4-0e4e-49d2-b503-41cd50ad3cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:52.862468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:52.868321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 681/1348] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:17:52.904340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:52.905875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:53.014910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b6c2-2931-40d5-a38d-03cd1bcf0327 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:53.064649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:53.091533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:53.092184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.092464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.092741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.094388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:53.210725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:53.212228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeac0f-128c-49fd-9596-38ca48b07722 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:53.233644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 668/1349] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:17:58.124296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111e8c4-bedc-4525-98a3-c3f2d719c1be None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.124824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111e8c4-bedc-4525-98a3-c3f2d719c1be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.124906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111e8c4-bedc-4525-98a3-c3f2d719c1be None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.126526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 682/1350] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:17:58.131594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbfe4-24e3-4462-b803-e02338f6e7bb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.132061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbfe4-24e3-4462-b803-e02338f6e7bb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.132181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbfe4-24e3-4462-b803-e02338f6e7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.264976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3e8fbfe4-24e3-4462-b803-e02338f6e7bb None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:58.266241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3e8fbfe4-24e3-4462-b803-e02338f6e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:58.289812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:58.391622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 669/1351] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:02.540995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb8c2-737b-422f-92e7-d13046514760 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:02.541583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb8c2-737b-422f-92e7-d13046514760 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:02.541583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb8c2-737b-422f-92e7-d13046514760 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:02.543077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 683/1352] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:02.549024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83f20-8ffe-4d0d-aad6-0b4f9de773d0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:02.549529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83f20-8ffe-4d0d-aad6-0b4f9de773d0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:02.549946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83f20-8ffe-4d0d-aad6-0b4f9de773d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:02.633609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-14d83f20-8ffe-4d0d-aad6-0b4f9de773d0 None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:02.634799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-14d83f20-8ffe-4d0d-aad6-0b4f9de773d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:02.709570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:02.751324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 670/1353] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:04.129826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a230ce-8fd3-4bdc-80b6-615883aa7077 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.130381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a230ce-8fd3-4bdc-80b6-615883aa7077 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.130381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a230ce-8fd3-4bdc-80b6-615883aa7077 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.131731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 684/1354] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:04.136764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97b67-4c9e-41ce-ab3f-fa87a0c0f20e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.137184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97b67-4c9e-41ce-ab3f-fa87a0c0f20e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.137301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97b67-4c9e-41ce-ab3f-fa87a0c0f20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.212733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3fa97b67-4c9e-41ce-ab3f-fa87a0c0f20e None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:04.213922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fa97b67-4c9e-41ce-ab3f-fa87a0c0f20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:04.322661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 671/1355] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:07.620157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475284-570b-4261-8e61-57efaed36410 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:07.620729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475284-570b-4261-8e61-57efaed36410 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:07.621123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475284-570b-4261-8e61-57efaed36410 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:07.624088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 685/1356] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:07.630256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf4c51-dd5b-4a0e-8f51-aedaf2a4eb7c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:07.631087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf4c51-dd5b-4a0e-8f51-aedaf2a4eb7c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:07.631533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf4c51-dd5b-4a0e-8f51-aedaf2a4eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:07.671694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:07.708025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-afcf4c51-dd5b-4a0e-8f51-aedaf2a4eb7c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:07.709512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-afcf4c51-dd5b-4a0e-8f51-aedaf2a4eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:07.823339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 672/1357] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:07.876495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:07.877077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:07.877630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:07.880504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 686/1358] 10.222.0.72 () {54 vars in 962 bytes} [Thu Aug 4 10:18:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:07.889859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2e7d-4b25-4cba-87f7-b0fe988e58bf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:07.890271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2e7d-4b25-4cba-87f7-b0fe988e58bf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:07.891175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2e7d-4b25-4cba-87f7-b0fe988e58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:07.976744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9c6d2e7d-4b25-4cba-87f7-b0fe988e58bf None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:07.977965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9c6d2e7d-4b25-4cba-87f7-b0fe988e58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.028514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:08.100522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 673/1359] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Aug 4 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:08.108700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:08.110284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.263379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175aa87-fb0f-425c-b7c3-62fb4b5e2410 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:08.346740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:08.390227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:08.390926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.391304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.391676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.393491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.488677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:08.490411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5b2df-f0a8-4500-93c4-eb382d515efa service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:08.510401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 687/1360] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:10.729812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.730217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.730486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.731789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 674/1361] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:10.737208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad03253-2aa3-44a5-b8bd-31e891982782 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.737669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad03253-2aa3-44a5-b8bd-31e891982782 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.737992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad03253-2aa3-44a5-b8bd-31e891982782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.814122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cad03253-2aa3-44a5-b8bd-31e891982782 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:10.815255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cad03253-2aa3-44a5-b8bd-31e891982782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:10.855529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.921116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 688/1362] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:10.968690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.970565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:11.142050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f169f3a-6730-4dff-8024-920d48892f12 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.152265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:11.287863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.288499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.288778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.289051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.290738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:11.387336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.388956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457c07-3694-4909-9d8a-a07e0cb47868 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.408492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 675/1363] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:13.814549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf82-83f1-4b7d-801f-25cd6d168854 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.815586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf82-83f1-4b7d-801f-25cd6d168854 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.816145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf82-83f1-4b7d-801f-25cd6d168854 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.817589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 689/1364] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:13.822998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ac5b-dece-4d19-845b-c9dd12327865 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.823446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ac5b-dece-4d19-845b-c9dd12327865 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.823623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ac5b-dece-4d19-845b-c9dd12327865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.839353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:13.897611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ff40ac5b-dece-4d19-845b-c9dd12327865 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:13.898866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff40ac5b-dece-4d19-845b-c9dd12327865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:14.008043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 676/1365] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:14.054954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.055312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.055676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.057111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 690/1366] 10.222.0.72 () {54 vars in 962 bytes} [Thu Aug 4 10:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:14.062374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6308-4e31-468a-bccd-bb5fb0cd75f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.062663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6308-4e31-468a-bccd-bb5fb0cd75f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.062937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6308-4e31-468a-bccd-bb5fb0cd75f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.156956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-014a6308-4e31-468a-bccd-bb5fb0cd75f9 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:14.158594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-014a6308-4e31-468a-bccd-bb5fb0cd75f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:14.200133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:14.292940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 677/1367] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Aug 4 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:14.304964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:14.306517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:14.460346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba6b7d-02df-4f16-a968-1768bb56645b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:14.482762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:14.583917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:14.584560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.584841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.585116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.586733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:14.682586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:14.684349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483e70e-e07f-4db8-b95a-aab9f451519d service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:14.703565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 691/1368] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:15.643455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1759e3-5cb6-4bab-a04b-eb7af7e35b7a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.644002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1759e3-5cb6-4bab-a04b-eb7af7e35b7a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.644002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1759e3-5cb6-4bab-a04b-eb7af7e35b7a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.646038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 678/1369] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:15.651163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2277d-f3d6-4aa1-9586-5af8baacf324 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.651607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2277d-f3d6-4aa1-9586-5af8baacf324 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.651704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2277d-f3d6-4aa1-9586-5af8baacf324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.685425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:15.721330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65e2277d-f3d6-4aa1-9586-5af8baacf324 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:15.723093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65e2277d-f3d6-4aa1-9586-5af8baacf324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:15.828542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 692/1370] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:16.170862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.170862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.171118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.172541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 679/1371] 10.222.0.72 () {54 vars in 964 bytes} [Thu Aug 4 10:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:16.177862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a6ced-72d4-4258-83b7-7d43741a77ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.178151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a6ced-72d4-4258-83b7-7d43741a77ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.178425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a6ced-72d4-4258-83b7-7d43741a77ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.267616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2f6a6ced-72d4-4258-83b7-7d43741a77ff None None] MFA Rules not processed for user `e60a8fe993054ee3a7563058956917d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:16.268935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2f6a6ced-72d4-4258-83b7-7d43741a77ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:16.317855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.350121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 693/1372] 10.222.0.72 () {58 vars in 1061 bytes} [Thu Aug 4 10:18:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:16.357964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.359519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:16.489770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25758451-6309-4948-830e-bf73c82b2d68 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.533814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.570448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.571095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.571377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.571649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.573278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:16.691732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.693202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032ff7d-cbc7-4986-bfcd-68091b19fb36 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.711551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 680/1373] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:18:16.817500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.818038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.818038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.819321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 694/1374] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:16.825285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aa967-ed48-4ceb-99af-32b0585cd652 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.825882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aa967-ed48-4ceb-99af-32b0585cd652 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.826265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569aa967-ed48-4ceb-99af-32b0585cd652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.909170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-569aa967-ed48-4ceb-99af-32b0585cd652 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:16.910873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-569aa967-ed48-4ceb-99af-32b0585cd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:16.937344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.044514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 681/1375] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:17.085146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.086708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:17.256089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b9703-5773-4738-8892-766fc421b3f8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.307532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.370169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.370819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.371105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.371380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.373055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:17.466308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.467987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea870-962c-433f-b381-450c8e3c60f3 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.485862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 695/1376] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:19.530461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.530984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.531080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.532534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 682/1377] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:19.537807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87a77-d142-45b0-852c-f6f87572ba87 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.538275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87a77-d142-45b0-852c-f6f87572ba87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.538374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87a77-d142-45b0-852c-f6f87572ba87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.603471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-77b87a77-d142-45b0-852c-f6f87572ba87 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:19.604769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77b87a77-d142-45b0-852c-f6f87572ba87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:19.612559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:19.713608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 696/1378] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:19.766424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:19.768003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:19.943657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5ddc-990a-4652-9941-a6320ebfc9f2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:19.947529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:20.063521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:20.064211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:20.064498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:20.064772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:20.067013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:20.176569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:20.178200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ed61-7f08-4dbe-8414-ac5e923a2638 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:20.197544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 683/1379] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:22.293243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.293819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.293819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.295167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 697/1380] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:22.302020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65a9d-44b0-4a8d-81d2-846bb86b0a52 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.302570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65a9d-44b0-4a8d-81d2-846bb86b0a52 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.302570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65a9d-44b0-4a8d-81d2-846bb86b0a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.317466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.383547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-50a65a9d-44b0-4a8d-81d2-846bb86b0a52 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:22.384804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50a65a9d-44b0-4a8d-81d2-846bb86b0a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:22.491997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 684/1381] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:22.531946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:22.533467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:22.684761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.696977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b093b3f-f468-4b65-8626-16b640c41d38 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:22.814220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:22.814748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.815025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.815296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.817461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:22.911651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:22.913271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c4018-97df-4cf2-b78e-63230cb2d1e6 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:22.924176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.933872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 698/1382] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:25.035717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.036484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.036484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.037701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 685/1383] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:18:25.043030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e993-dac6-44e3-9ce9-c6a66224143a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.043509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e993-dac6-44e3-9ce9-c6a66224143a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.043658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e993-dac6-44e3-9ce9-c6a66224143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.109254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3453e993-dac6-44e3-9ce9-c6a66224143a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:25.110538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3453e993-dac6-44e3-9ce9-c6a66224143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:25.206436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 699/1384] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:25.250387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.251906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:25.337725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.403312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17972c13-bd94-4ed5-b150-a420712e18c1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.518130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.518758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.519044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.520065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.521566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:25.572865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.616298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.617812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82985-d3d6-4942-9682-9e6810e633a1 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.636476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 686/1385] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:28.016665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.017202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.017202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.018567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 700/1386] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:28.023919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e7670-2e26-4e5d-a69e-d0b4dfbb72c9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.024359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e7670-2e26-4e5d-a69e-d0b4dfbb72c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.024500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e7670-2e26-4e5d-a69e-d0b4dfbb72c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.091908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ea2e7670-2e26-4e5d-a69e-d0b4dfbb72c9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:28.093147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ea2e7670-2e26-4e5d-a69e-d0b4dfbb72c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:28.190654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 687/1387] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:28.234861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.236458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:28.317933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.387244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147cb8-a382-449e-a038-2291b6e9d605 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.498843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.499454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.499738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.500036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.501784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:28.539675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.591327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.592887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31059455-a03d-4093-bc25-873231cbdabb service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.611335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 701/1388] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:30.663409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.664058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.664058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.665339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 688/1389] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:30.670603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baceba-94ad-4246-a878-5c59e4162cda None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.671056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baceba-94ad-4246-a878-5c59e4162cda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.671170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baceba-94ad-4246-a878-5c59e4162cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.740591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-38baceba-94ad-4246-a878-5c59e4162cda None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:30.741790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-38baceba-94ad-4246-a878-5c59e4162cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:30.836506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 702/1390] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:30.881378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.882874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:30.915389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:31.033818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091bf2b-266e-4a0e-8ec9-ded30d7e8341 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:31.137116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:31.149195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:31.149811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.150093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.150363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.152024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:31.241886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:31.243645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcdda3-3512-49f7-b187-36d4b1339dc8 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:31.262394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 689/1391] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:31.944070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2f702-c78c-489f-9c22-e4175bd34cd9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.944653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2f702-c78c-489f-9c22-e4175bd34cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.944653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2f702-c78c-489f-9c22-e4175bd34cd9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.945999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 703/1392] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:31.951275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacd536-dfd2-4270-b7f5-6ffdc6666606 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.951717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacd536-dfd2-4270-b7f5-6ffdc6666606 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.951917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacd536-dfd2-4270-b7f5-6ffdc6666606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.030691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5eacd536-dfd2-4270-b7f5-6ffdc6666606 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:32.032499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5eacd536-dfd2-4270-b7f5-6ffdc6666606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.107750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.146044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 690/1393] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:32.201618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.201846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.202116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.203446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 704/1394] 10.222.0.72 () {54 vars in 966 bytes} [Thu Aug 4 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:32.209133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9d86-5177-4e10-8964-a194d1a326f8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.209485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9d86-5177-4e10-8964-a194d1a326f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.209766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9d86-5177-4e10-8964-a194d1a326f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.287180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-370d9d86-5177-4e10-8964-a194d1a326f8 None None] MFA Rules not processed for user `5f0130283f8f42bb94a894b5a1b55d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:32.288412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-370d9d86-5177-4e10-8964-a194d1a326f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.367939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 691/1395] 10.222.0.72 () {58 vars in 1063 bytes} [Thu Aug 4 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:32.375293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.376842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.391029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.495060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78e2b-2f1e-4653-94bd-033dc96f1135 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.568588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.569196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.569471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.569737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.571321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.637501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.678919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:32.680665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338962a-80c1-4534-9363-bf0178d783be service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:32.698988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 705/1396] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:33.319616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a015f8-ad68-40fc-b960-712bb32c87df None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.320250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a015f8-ad68-40fc-b960-712bb32c87df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.320250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a015f8-ad68-40fc-b960-712bb32c87df None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.321622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 692/1397] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:33.327547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.328134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.328246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.398060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:33.399278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.438209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9cda-31d2-4b63-bfe1-ee8185282c4a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.438482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9cda-31d2-4b63-bfe1-ee8185282c4a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.438766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9cda-31d2-4b63-bfe1-ee8185282c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.499893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 706/1398] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:33.506259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3d9c9cda-31d2-4b63-bfe1-ee8185282c4a None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:33.507498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3d9c9cda-31d2-4b63-bfe1-ee8185282c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.516238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.544572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.545926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.572944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.599710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 693/1399] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:33.651266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b2914-a925-4c3a-a36b-93a9172a8464 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.651555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b2914-a925-4c3a-a36b-93a9172a8464 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.651834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b2914-a925-4c3a-a36b-93a9172a8464 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.653206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 694/1400] 10.222.0.72 () {54 vars in 943 bytes} [Thu Aug 4 10:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:18:33.659255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.659540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.659849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.665531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48934276-c8be-4bef-a368-21f283c0e7cb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.736497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] MFA Rules not processed for user `6cc87ba3c4b9425d91a293211fbc7df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:33.736497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.779063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.779943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.780459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.780734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.782340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.801976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.824349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.854995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 695/1401] 10.222.0.72 () {58 vars in 1040 bytes} [Thu Aug 4 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:33.865531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.875158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ade4d-14f3-4e04-a8f9-712f73bbf49b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.877619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.879280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5c0ee-7e69-4f0d-b149-e0124fc9d788 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.902172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 707/1402] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:34.110013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.110642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.110921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.111194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.113490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:34.202128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.203721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e1cfe-7c56-487e-95fd-7d714f918f82 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.212007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 696/1403] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 10:18:35.958896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.959467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.959467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.964387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 708/1404] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:35.966097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775af192-9b84-4c7c-8e3b-a8c860f111b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.966516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775af192-9b84-4c7c-8e3b-a8c860f111b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.966748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775af192-9b84-4c7c-8e3b-a8c860f111b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.979670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:36.043370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-775af192-9b84-4c7c-8e3b-a8c860f111b5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:36.044793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-775af192-9b84-4c7c-8e3b-a8c860f111b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:36.150327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 697/1405] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:36.195294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:36.198067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:36.359359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:36.362352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d728f0-4a62-4460-8d93-8ade09ad67ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:36.490853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:36.491472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.491748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.492051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.493924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:36.593035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:36.594671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb98a2-c424-4208-8821-3a540f1a65d6 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:36.614088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 709/1406] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:38.664593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.665159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.665159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.666627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 698/1407] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:38.671820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba936e0-5868-433e-91a8-ea1816eb52f1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.672336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba936e0-5868-433e-91a8-ea1816eb52f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.672451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba936e0-5868-433e-91a8-ea1816eb52f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.677658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:38.746731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6ba936e0-5868-433e-91a8-ea1816eb52f1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:38.748388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6ba936e0-5868-433e-91a8-ea1816eb52f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:38.851268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 710/1408] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:38.900462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.901975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:38.995032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.067609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ebfcb-8b0f-4f1f-baad-2c0bae8277bc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.194560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.195478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.195945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.196268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.197892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:39.256741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.303335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:39.305453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98005ac-d6d5-4d11-8484-009b2b667a68 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:39.326339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 699/1409] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:18:48.849383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=64 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.323620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=23 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.412114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7205d-da78-4d23-b6e7-705ea057d06d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.412670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7205d-da78-4d23-b6e7-705ea057d06d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.412670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7205d-da78-4d23-b6e7-705ea057d06d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.414192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 711/1410] 10.222.0.72 () {54 vars in 918 bytes} [Thu Aug 4 10:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:56.419107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977434-3fe1-472e-b6d0-e4d67e591187 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.419518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977434-3fe1-472e-b6d0-e4d67e591187 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.419651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977434-3fe1-472e-b6d0-e4d67e591187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.493192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a5977434-3fe1-472e-b6d0-e4d67e591187 None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:56.494372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5977434-3fe1-472e-b6d0-e4d67e591187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:56.597038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 700/1411] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Aug 4 10:18:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:56.614621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ea8d1-435a-43c1-a65d-600eeaa5f252 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.614906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ea8d1-435a-43c1-a65d-600eeaa5f252 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.615177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ea8d1-435a-43c1-a65d-600eeaa5f252 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.616542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 712/1412] 10.222.0.72 () {54 vars in 918 bytes} [Thu Aug 4 10:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:56.621193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59aba-3ac6-43ae-a79e-506a553ebaa7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.621475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59aba-3ac6-43ae-a79e-506a553ebaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.621742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59aba-3ac6-43ae-a79e-506a553ebaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.692330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.695537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-34f59aba-3ac6-43ae-a79e-506a553ebaa7 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:56.697311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-34f59aba-3ac6-43ae-a79e-506a553ebaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:56.809955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 701/1413] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Aug 4 10:18:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:18:59.270775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.271274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.271379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.272718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:18:59.277834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8a21-76a6-4a3b-b7a5-834818172d44 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.278256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8a21-76a6-4a3b-b7a5-834818172d44 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.278386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8a21-76a6-4a3b-b7a5-834818172d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.342822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5acd8a21-76a6-4a3b-b7a5-834818172d44 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:59.344164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5acd8a21-76a6-4a3b-b7a5-834818172d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:59.400080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.441949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:18:59.485182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:59.486810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:59.630949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b090f78-4183-4d8c-aef1-075993daa068 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.701572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.746100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:59.746722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.747002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.747278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.748925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:59.838125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:59.839724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96f833-98eb-4c40-bb4c-0a10c1861334 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:59.857842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 714/1416] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:00.760922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.761502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.761502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.762841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:00.768376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8c97-92f0-4e30-9b6e-6a6f75dc4d04 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.768832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8c97-92f0-4e30-9b6e-6a6f75dc4d04 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.768941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8c97-92f0-4e30-9b6e-6a6f75dc4d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.842729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d5fa8c97-92f0-4e30-9b6e-6a6f75dc4d04 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:00.844032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d5fa8c97-92f0-4e30-9b6e-6a6f75dc4d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:00.870209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.938050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:01.094237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.095735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:01.237933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fd005-20a9-4289-91c6-ab933a07b5c6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.278285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.351298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.351833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.352147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.352424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.354046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:01.441314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.442879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b511d25-d5b8-44e9-b226-76c5be809f26 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.461071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 704/1419] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:02.338348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.338923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.338923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.340348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:02.345497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b5e7-647a-423e-ab79-2b73a5d98780 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.345922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b5e7-647a-423e-ab79-2b73a5d98780 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.346048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b5e7-647a-423e-ab79-2b73a5d98780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.400382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.424972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ab2b5e7-647a-423e-ab79-2b73a5d98780 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:02.426166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ab2b5e7-647a-423e-ab79-2b73a5d98780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:02.518829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:19:02.672658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.674688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:02.816594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2b089-b9bf-4cf2-9389-b8bc2e6208c3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.828159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.929537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.930147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.930427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.930696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.932331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:03.019930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:03.021535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2dbb1-be1e-43c3-8313-f3cb6d2bfef8 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:03.038941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 717/1422] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:03.900922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:03.901547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:03.901547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:03.902806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:03.908315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3524e7-e07b-46b9-97c4-7c68c707cdae None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:03.908706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3524e7-e07b-46b9-97c4-7c68c707cdae None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:03.908867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3524e7-e07b-46b9-97c4-7c68c707cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:03.929273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:03.985672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2d3524e7-e07b-46b9-97c4-7c68c707cdae None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:03.986891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2d3524e7-e07b-46b9-97c4-7c68c707cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:04.085536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:04.239754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:04.241232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:04.352046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:04.392591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd2793-f8a2-4290-8868-548ac105223f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:04.519241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:04.519886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:04.520195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:04.520472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:04.522077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:04.598134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:04.611716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:04.613327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0379d0-be19-4d65-8528-5014452a543a service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:04.631183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 707/1425] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:05.483049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.483586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.483586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.484987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:05.490148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba6e4-46d6-4bd3-b861-bef1338b8346 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.490560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba6e4-46d6-4bd3-b861-bef1338b8346 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.490679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba6e4-46d6-4bd3-b861-bef1338b8346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.565258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f63ba6e4-46d6-4bd3-b861-bef1338b8346 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:05.566456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f63ba6e4-46d6-4bd3-b861-bef1338b8346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:05.661325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:05.809959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:05.811468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:05.893495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:05.954161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c26e5-a8f5-4df7-a385-a8fd926b86e4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.062171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.062773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.063049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.063315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.065545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:06.111602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.155858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.157526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d33f-c256-440c-80ca-0454a77cced2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.175286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 720/1428] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:06.989176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.989720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.989720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.991070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:06.996122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcccb-4d3f-46e1-ab32-1d3ebbf79088 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.996527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcccb-4d3f-46e1-ab32-1d3ebbf79088 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.996674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcccb-4d3f-46e1-ab32-1d3ebbf79088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.081390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-663bcccb-4d3f-46e1-ab32-1d3ebbf79088 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:07.082594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-663bcccb-4d3f-46e1-ab32-1d3ebbf79088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:07.176746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:07.328095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.329639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:07.378095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.473478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffeb16f-e693-4a61-b00c-50c3cdb37f8a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.582978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.583586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.583864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.584165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.585794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:07.592564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.674614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.676279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e692c-b232-41a5-87ec-3b3e490686f2 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.695877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 710/1431] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:08.503232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:08.503752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:08.503752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:08.505107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:08.510044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd50146-e48b-4df8-9864-4ac4b889799f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:08.510475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd50146-e48b-4df8-9864-4ac4b889799f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:08.510592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd50146-e48b-4df8-9864-4ac4b889799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:08.584502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bbd50146-e48b-4df8-9864-4ac4b889799f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:08.585708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bbd50146-e48b-4df8-9864-4ac4b889799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:08.651658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:08.680922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:08.831110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:08.832671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:08.851520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:08.977900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2b9f0-fc10-4c8d-b38e-e159733ecfa6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:09.063213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:09.089135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:09.089677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:09.089966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:09.090222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:09.091864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:09.180482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:09.182052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc44268-d52b-40a4-86d0-9bfb22e49bfe service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:09.199917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 723/1434] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:10.008548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.009223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.009223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.010507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:10.015732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082caa-3a45-4539-9bce-97d680b2150f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.016218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082caa-3a45-4539-9bce-97d680b2150f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.016314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082caa-3a45-4539-9bce-97d680b2150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.090469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3e082caa-3a45-4539-9bce-97d680b2150f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:10.091664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e082caa-3a45-4539-9bce-97d680b2150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:10.132597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.186008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:10.334310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.335711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:10.491286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7d97a-2b7b-4cb8-b3c1-078bf887b231 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.565056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.624241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.624782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.625060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.625331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.626981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:10.715484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:10.717085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff952e06-fc2a-491a-878b-b4747b898a31 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:10.735005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 713/1437] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:11.537454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.537980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.537980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.539345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:11.544235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb812d48-9a9a-4b02-8757-b55a81cb463e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.544676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb812d48-9a9a-4b02-8757-b55a81cb463e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.544772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb812d48-9a9a-4b02-8757-b55a81cb463e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.619700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fb812d48-9a9a-4b02-8757-b55a81cb463e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:11.620927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fb812d48-9a9a-4b02-8757-b55a81cb463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:11.622878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.716102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:11.865227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.866690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:12.012206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1e1aa-367c-4853-95ac-7c2b804bffc6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.039302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.123803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.124360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.124638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.124906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.127004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:12.217440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.219050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa691639-d77b-4794-9356-f63c1ffd5094 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.236906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 726/1440] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:14.147024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.147598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.147598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.148890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:14.154133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989040-2b57-4790-88d6-208f71e895f6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.154595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989040-2b57-4790-88d6-208f71e895f6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.154813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989040-2b57-4790-88d6-208f71e895f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.195341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.220364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e2989040-2b57-4790-88d6-208f71e895f6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:14.221579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e2989040-2b57-4790-88d6-208f71e895f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:14.316461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:14.354540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.356068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:14.458998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34243d7-df60-48bb-8b62-e82fbeb3689d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.490043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.532296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.532907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.533189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.533464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.535074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:14.643362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.644822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffc31-d622-4c42-80b6-e9b4459f9034 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.662643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 716/1443] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:16.873172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.873844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.873844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.875070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:16.880370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1ca9-54da-44f9-86c8-8a4d0fdac89c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.880802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1ca9-54da-44f9-86c8-8a4d0fdac89c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.880923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1ca9-54da-44f9-86c8-8a4d0fdac89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.946022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-427b1ca9-54da-44f9-86c8-8a4d0fdac89c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:16.947209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-427b1ca9-54da-44f9-86c8-8a4d0fdac89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:16.960045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.043140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:17.081059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:17.082599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:17.183110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec5e4-06a1-470d-8178-0e1f4e316737 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.194916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.256303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.256921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.257200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.257468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.259080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:17.372623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:17.374191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942dd12-9868-40fd-8ad5-13adbc8043f6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:17.392930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 729/1446] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:19.712509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.713074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.713074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.714399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:19.719870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d29691-79e2-4377-b0e3-e3171af3bc2c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.720327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d29691-79e2-4377-b0e3-e3171af3bc2c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.720469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d29691-79e2-4377-b0e3-e3171af3bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.772605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.791247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c6d29691-79e2-4377-b0e3-e3171af3bc2c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:19.792499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6d29691-79e2-4377-b0e3-e3171af3bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:19.887347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:19.926689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.928509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:20.036169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd73f-98f5-4920-8ce8-469015e2c066 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.076280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.109165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.110282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.110553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.110816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.112434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:20.218638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.220695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a000ec-aabb-48d3-90a8-1240569cc09b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.238794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 719/1449] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:22.374480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.375609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.375609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.376954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:22.382265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc87ad-ed13-4f06-beba-b49fe0c798a6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.382689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc87ad-ed13-4f06-beba-b49fe0c798a6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.382793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc87ad-ed13-4f06-beba-b49fe0c798a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.447607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b1cc87ad-ed13-4f06-beba-b49fe0c798a6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:22.448825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b1cc87ad-ed13-4f06-beba-b49fe0c798a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:22.470360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.545128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:22.584422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.585958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:22.688331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3ebe3-52fe-4a61-8525-7008ba641098 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.707741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.761150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.761764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.762045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.762315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.763933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:22.870420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.871785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb53b-51fd-4eb9-8eeb-22f776a70707 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.889177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 732/1452] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:24.950069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.950609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.950705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.952025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:24.957999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e3db1-2830-4d25-893c-515d16d4e8ed None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.958405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e3db1-2830-4d25-893c-515d16d4e8ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.958558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e3db1-2830-4d25-893c-515d16d4e8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.021379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.024257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e80e3db1-2830-4d25-893c-515d16d4e8ed None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:25.025790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e80e3db1-2830-4d25-893c-515d16d4e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:25.123881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:25.163063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.164590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:25.266082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1698e-50c8-4f70-b33a-358c9c69d25a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.314484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.339316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.339924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.340232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.340507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.342105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:25.449881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.451317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe9ebf-0b1f-43cc-b793-feaa0ad6a35d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.469994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 722/1455] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:27.507989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.508553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.508553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.509787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:27.515794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adcf7d-2f5a-419e-bf22-75b7d3f15f12 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.516280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adcf7d-2f5a-419e-bf22-75b7d3f15f12 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.516387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adcf7d-2f5a-419e-bf22-75b7d3f15f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.582872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-20adcf7d-2f5a-419e-bf22-75b7d3f15f12 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:27.584117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-20adcf7d-2f5a-419e-bf22-75b7d3f15f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:27.614349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.682009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:27.725513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.727065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:27.830111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc638e1-ca6d-462d-a252-80689db85cfb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.859606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.904605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.905227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.905510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.906289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.907905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:28.015280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:28.016896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563ca8f-cb2b-4f4d-8bf8-9ff91f4c6c3b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:28.035098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 735/1458] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:30.129353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.129920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.129920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.131217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:30.136359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69edf-cb66-48ed-b8d3-ecd056e30cb9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.136759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69edf-cb66-48ed-b8d3-ecd056e30cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.136915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69edf-cb66-48ed-b8d3-ecd056e30cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.201122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-93e69edf-cb66-48ed-b8d3-ecd056e30cb9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:30.202312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-93e69edf-cb66-48ed-b8d3-ecd056e30cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:30.210894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.298505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:30.336885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.338392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:30.439995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b4aef-9724-451a-8819-9b50e38ce479 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.492787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.511135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.511741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.512049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.512326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.513925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:30.620030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.621469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9222712-d30c-4c57-ba11-49b88a5a15d1 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.639713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 725/1461] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:32.681946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.682508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.682508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.683893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:32.690034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36298c-8104-46cb-9e11-f17cfa0c20ef None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.690487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36298c-8104-46cb-9e11-f17cfa0c20ef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.690584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36298c-8104-46cb-9e11-f17cfa0c20ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.755403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8d36298c-8104-46cb-9e11-f17cfa0c20ef None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:32.756636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8d36298c-8104-46cb-9e11-f17cfa0c20ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:32.793276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.851982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:32.889994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.891551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:32.993953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a382e-9516-4c42-8819-b3eb7358fb17 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.152145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.188540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.189146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.189419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.189686 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.191266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:33.297788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.299218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e85dc6-7608-491a-b3f6-9249bd9c705f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.317111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 738/1464] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:35.450666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.451223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.451223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.452604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:35.457994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06af27-7cba-4602-91af-778098775fdf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.458447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06af27-7cba-4602-91af-778098775fdf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.458553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06af27-7cba-4602-91af-778098775fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.522880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0d06af27-7cba-4602-91af-778098775fdf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:35.524117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0d06af27-7cba-4602-91af-778098775fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:35.542189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:35.620372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:19:35.657432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.659766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:35.783881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e013f0-fd64-4eb9-844b-9e9f9a2894e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:35.848833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:35.855753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:35.856400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.856683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.856955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.858602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:35.965664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:35.967094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850e043-a381-4694-915a-5188c645d743 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:35.985209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 728/1467] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:19:37.982881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.983944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.983944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.985348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:37.990683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92068d-cae0-47cc-bb74-c71358b97084 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.991099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92068d-cae0-47cc-bb74-c71358b97084 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.991230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92068d-cae0-47cc-bb74-c71358b97084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.057278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ae92068d-cae0-47cc-bb74-c71358b97084 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:38.058486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae92068d-cae0-47cc-bb74-c71358b97084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:38.104995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.153452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:38.191183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.192845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:38.295419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37663d4-954f-4f9b-a564-72006c203492 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.343932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.370191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.370835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.371115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.371386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.373034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:38.477092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.478515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69150515-107c-4627-8255-e0f522693cb3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.496857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 741/1470] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:19:48.151596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=22 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.494672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=66 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.155339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a558-1c91-4b4e-be06-b4352db53c1b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.155926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a558-1c91-4b4e-be06-b4352db53c1b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.156073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a558-1c91-4b4e-be06-b4352db53c1b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.157324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 730/1471] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:58.162726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c4ad-c34d-4479-8460-19b209e37a42 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.163195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c4ad-c34d-4479-8460-19b209e37a42 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.163304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c4ad-c34d-4479-8460-19b209e37a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.227740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b247c4ad-c34d-4479-8460-19b209e37a42 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:58.229004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b247c4ad-c34d-4479-8460-19b209e37a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:58.324910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 742/1472] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:58.409373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.409679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.409998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.411327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 731/1473] 10.222.0.72 () {54 vars in 966 bytes} [Thu Aug 4 10:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:19:58.416433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77237-0394-4469-9f2f-00b7b69fba9c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.416720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77237-0394-4469-9f2f-00b7b69fba9c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.417013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c77237-0394-4469-9f2f-00b7b69fba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.495552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-84c77237-0394-4469-9f2f-00b7b69fba9c None None] MFA Rules not processed for user `5f0130283f8f42bb94a894b5a1b55d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:58.496773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84c77237-0394-4469-9f2f-00b7b69fba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:58.517555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.571030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 743/1474] 10.222.0.72 () {58 vars in 1063 bytes} [Thu Aug 4 10:19:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:19:58.579395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.580961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:58.682027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107c4e9-d0ef-47f8-b2f2-9040f24f7ec8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.754760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.755363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.755640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.755911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.757530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:58.811936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.866354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.867812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19567e-2ace-4be7-b0a6-e69bb2ec1430 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.886626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 732/1475] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:00.984505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.985097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.985097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.986456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 744/1476] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:00.991518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8254-b40e-4734-8aa3-a038a7492e87 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.991895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8254-b40e-4734-8aa3-a038a7492e87 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.992107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8254-b40e-4734-8aa3-a038a7492e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.058051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a4fd8254-b40e-4734-8aa3-a038a7492e87 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:01.060293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4fd8254-b40e-4734-8aa3-a038a7492e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:01.154804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 733/1477] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:20:01.177979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:01.179482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:01.302310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf36d-1f46-4999-a72b-35bfb59aac79 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:01.349884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:01.393035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:01.393644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.393924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.394195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.394875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:01.396632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:20:01.396632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:20:01.396632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:20:01.397795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a397e65-8ce5-4a9e-a590-90be71218a1a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:01.418139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 745/1478] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Aug 4 10:20:01 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:20:03.427542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.428134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.428134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.429482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 734/1479] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:03.435369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789277-7525-4171-83ac-9cdd3eba5575 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.435796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789277-7525-4171-83ac-9cdd3eba5575 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.435919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789277-7525-4171-83ac-9cdd3eba5575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.500004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b789277-7525-4171-83ac-9cdd3eba5575 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:03.501108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b789277-7525-4171-83ac-9cdd3eba5575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.593993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 746/1480] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:20:03.637434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.639523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.649190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.784067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9252e-2971-4995-9d12-a3f8dcfd1dd7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:03.864550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.899091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:03.900222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.900693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.901372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.904758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.993408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:03.994989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c0ad-da37-458f-a65a-ba24f78878c7 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.014365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 735/1481] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:06.191314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.191879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.191879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.193262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 747/1482] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:06.198464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af7a4-6465-4b7c-9ffb-dfb789f7ad8f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.198860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af7a4-6465-4b7c-9ffb-dfb789f7ad8f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.199011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af7a4-6465-4b7c-9ffb-dfb789f7ad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.268802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d18af7a4-6465-4b7c-9ffb-dfb789f7ad8f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:06.270003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d18af7a4-6465-4b7c-9ffb-dfb789f7ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:06.315511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.364887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 736/1483] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:20:06.401896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:06.404040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:06.427392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.504418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505df13-1d74-49e6-9aa7-5b25cbadd8f3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:06.578468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:06.579082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.579358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.579626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.581283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:06.660588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.695591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:06.697056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ed3d4-3f90-4d95-9ab8-a3771317a49b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:06.715121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 748/1484] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:08.765664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.766263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.766263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.767547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 737/1485] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:08.772766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b15ba-4177-44fd-8635-41e39ade7cd0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.773193 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b15ba-4177-44fd-8635-41e39ade7cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.773365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b15ba-4177-44fd-8635-41e39ade7cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.837173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4c9b15ba-4177-44fd-8635-41e39ade7cd0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:08.838477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c9b15ba-4177-44fd-8635-41e39ade7cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:08.933930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 749/1486] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:20:08.972831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:08.974340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:09.078587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e074e-bbd8-4697-afe5-90c11a8f2a14 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.149907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:09.151658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.152335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.152790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.153069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.155052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:09.263749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:09.265230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e3bbe-af93-4717-a157-7d0f65c1bef2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:09.282917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 738/1487] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:11.429344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.429895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.429895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.431225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 750/1488] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:11.436876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb773-b427-48c6-a502-4fa4b1de53c1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.437302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb773-b427-48c6-a502-4fa4b1de53c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.437440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb773-b427-48c6-a502-4fa4b1de53c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.503830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-548cb773-b427-48c6-a502-4fa4b1de53c1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:11.504980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-548cb773-b427-48c6-a502-4fa4b1de53c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:11.557537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.602620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 739/1489] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:20:11.643480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.645037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:11.680325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.747238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1366884-43e8-44b5-8e47-17f8e9ef986e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.817687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.818289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.818591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.818860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.820485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:11.898675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.927478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.928931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b8463-fc1a-4136-9fef-17fa80e047ec service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.946277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 751/1490] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:14.170082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.170663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.170663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.171998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 740/1491] 10.222.0.72 () {54 vars in 927 bytes} [Thu Aug 4 10:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:20:14.177573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e9821-acb7-4004-afeb-a3ed7effdd40 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.178010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e9821-acb7-4004-afeb-a3ed7effdd40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.178132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e9821-acb7-4004-afeb-a3ed7effdd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.245112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b2e9821-acb7-4004-afeb-a3ed7effdd40 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:14.246328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b2e9821-acb7-4004-afeb-a3ed7effdd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:14.342791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 752/1492] 10.222.0.72 () {58 vars in 1024 bytes} [Thu Aug 4 10:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:20:14.385308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:14.386863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:14.487586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e078a-b560-4c0d-954c-66b617a76a56 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:14.560140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:14.561117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.561401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.561674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.563290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:14.572428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:14.671843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:14.673255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210588d-78bd-40fb-814c-bcb6f096b291 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:14.692388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 741/1493] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:20:24.340392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=82 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:24.690036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=50 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:25.095973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:25.096588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:25.096588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:25.169966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:25.171148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:25.266695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 753/1494] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:21:25.274938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:25.275431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:25.275538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:25.350554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:25.352225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:25.447390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 742/1495] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:21:25.452026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:25.453222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:25.465028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65218e-8e72-42fb-9f2b-d25f0b63c533 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:25.524978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:25.561010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:25.561628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:25.561907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:25.562176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:25.562871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:25.563197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:25.564243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:25.564243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:21:25.564243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:21:25.565125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:25.576099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 754/1496] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:25.584457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:25.594005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62d8a3-ab7c-4ec2-a1cf-ef15b19a6205 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:25.653592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:25.689906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:25.690546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:25.690821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:25.691087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:25.691739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:25.693487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:25.693487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:21:25.693487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:21:25.694682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:25.749377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 743/1497] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:21:25 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:21:25.753574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:25.755126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:25.788401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c54f9d-a78c-42b7-b6a0-a0bd6f547346 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:25.880697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:25.881302 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:25.881576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:25.881842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:25.882501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:25.883532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:25.883532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:21:25.883532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:21:25.884808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:25.926588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 755/1498] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:21:25 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:21:25.931365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:25.933385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:25.999856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9e65-fc9d-435c-a914-5b5278e151e3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:26.083041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:26.094869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:26.095485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.095764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.096068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.096760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:26.097764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:26.097764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:21:26.097764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:21:26.098643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:26.111291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 744/1499] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:21:26.115472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:26.117031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:26.128970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bab3f4-a58d-4f2c-875e-be69a687362c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:26.153084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:26.222717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:26.223248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.223528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.223800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.224526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:26.225532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:26.225532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:21:26.225532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:21:26.227009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b3a96-9e3b-4cdf-81a2-038140210a0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:26.239537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 756/1500] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:21:26.243939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:26.245497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:26.279823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18055377-2384-468c-b337-244eb9eeb509 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:26.371702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:26.372388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.372670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.372947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] PATH_INFO: `/v3/projects/faaed6f37dea4b3e9be01486e92101f7/users/253d3533bab043a5b24cd66371e218ba/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.394008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faaed6f37dea4b3e9be01486e92101f7, user_id=253d3533bab043a5b24cd66371e218ba, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:26.395035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:26.395035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:21:26.395035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:21:26.397030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51cb81-b3ba-480d-b94a-20cec8927308 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:26.421522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 745/1501] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:21:26 2022] PUT /identity/v3/projects/faaed6f37dea4b3e9be01486e92101f7/users/253d3533bab043a5b24cd66371e218ba/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:21:26.433994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7f31-02ab-4ab1-898c-d32f316de03e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.434286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7f31-02ab-4ab1-898c-d32f316de03e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.434568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7f31-02ab-4ab1-898c-d32f316de03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.489313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-313c7f31-02ab-4ab1-898c-d32f316de03e None None] MFA Rules not processed for user `253d3533bab043a5b24cd66371e218ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:26.490448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-313c7f31-02ab-4ab1-898c-d32f316de03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:26.565797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 757/1502] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:21:26.579079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.579368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.579646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.581020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 746/1503] 10.222.0.72 () {54 vars in 964 bytes} [Thu Aug 4 10:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:21:26.586505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662880d-7f7d-42aa-8b0e-a0ebf3e812fc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.586792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662880d-7f7d-42aa-8b0e-a0ebf3e812fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.587061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662880d-7f7d-42aa-8b0e-a0ebf3e812fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.617689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:26.662189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0662880d-7f7d-42aa-8b0e-a0ebf3e812fc None None] MFA Rules not processed for user `e60a8fe993054ee3a7563058956917d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:26.663399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0662880d-7f7d-42aa-8b0e-a0ebf3e812fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:26.736538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 758/1504] 10.222.0.72 () {58 vars in 1061 bytes} [Thu Aug 4 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:21:26.744508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:26.745938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:26.794637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:26.848898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331e200-17c2-4b98-9976-d5d0a4b81c12 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:26.921673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:26.922283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:26.922561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:26.922832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:26.924747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:27.021398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:27.022824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:27.026258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:27.041389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 747/1505] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:21:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:21:27.086298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcec4-70c6-4266-bb94-73a0d9924f54 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:27.086588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcec4-70c6-4266-bb94-73a0d9924f54 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:27.086867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcec4-70c6-4266-bb94-73a0d9924f54 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:27.088249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 759/1506] 10.222.0.72 () {52 vars in 830 bytes} [Thu Aug 4 10:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:21:27.091752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:27.092977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:27.190699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c45-51cb-4254-9426-033f944920e9 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:27.290505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:27.291210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:27.291351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:27.291627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] PATH_INFO: `/v3/users/253d3533bab043a5b24cd66371e218ba` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:27.308972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253d3533bab043a5b24cd66371e218ba)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:27.310028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:27.310028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:21:27.310028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:21:27.311267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:27.326876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:27.382354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253d3533bab043a5b24cd66371e218ba'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:27.382813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253d3533bab043a5b24cd66371e218ba'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:27.383340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253d3533bab043a5b24cd66371e218ba'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:27.405741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253d3533bab043a5b24cd66371e218ba'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:27.419766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253d3533bab043a5b24cd66371e218ba'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:27.435135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 748/1507] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:21:27 2022] DELETE /identity/v3/users/253d3533bab043a5b24cd66371e218ba => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:21:27.439671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:27.440155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:27.440327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:27.496126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:27.497329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:27.572164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:27.585517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 760/1508] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:21:27.595780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:27.597087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:27.696667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f70d97-15f1-4d1f-a398-cdc9cf3698a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:27.749223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:27.770637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:27.771250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:27.771525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:27.771791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:27.773902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:27.879031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:27.880456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f12b2-0164-401a-9a57-eb29a14803bb service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:27.898690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 749/1509] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:21:28.223813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:28.225081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:28.237126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc60a55-ad06-4886-a475-6dccea435442 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:28.335413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:28.336160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:28.336441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:28.336713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] PATH_INFO: `/v3/projects/faaed6f37dea4b3e9be01486e92101f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:28.346626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faaed6f37dea4b3e9be01486e92101f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:28.347642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:21:28.347642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:21:28.347642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:21:28.348862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:28.404480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faaed6f37dea4b3e9be01486e92101f7', 'request_id': 'req-69d48968-575c-4f1c-bef1-06e4916ce3e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:28.410236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faaed6f37dea4b3e9be01486e92101f7', 'request_id': 'req-69d48968-575c-4f1c-bef1-06e4916ce3e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:28.419881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faaed6f37dea4b3e9be01486e92101f7', 'request_id': 'req-69d48968-575c-4f1c-bef1-06e4916ce3e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:28.449920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] The token cache is being invalidate because project faaed6f37dea4b3e9be01486e92101f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:21:28.450507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:28.450697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-69d48968-575c-4f1c-bef1-06e4916ce3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:21:28.452017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 761/1510] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:21:28 2022] DELETE /identity/v3/projects/faaed6f37dea4b3e9be01486e92101f7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:21:37.895949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=62 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:38.447933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:04.771228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:04.771851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:04.771851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:04.773216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:04.778069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c305cc1-54de-40c0-95d8-6f8699ea003f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:04.778478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c305cc1-54de-40c0-95d8-6f8699ea003f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:04.778621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c305cc1-54de-40c0-95d8-6f8699ea003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:04.858053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9c305cc1-54de-40c0-95d8-6f8699ea003f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:04.859259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c305cc1-54de-40c0-95d8-6f8699ea003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:04.962361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:05.118446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:05.120563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:05.237982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3c64-e72d-450e-983e-f6d48dd96b05 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:05.244230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:05.244857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:05.245141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:05.245535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:05.254553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:05.256190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca88c-4b9e-4340-b672-a82dc6283462 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:05.265676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 751/1513] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:06.104268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fdbf-8397-4533-8e7c-5a7a886886be None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:06.104819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fdbf-8397-4533-8e7c-5a7a886886be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:06.104819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fdbf-8397-4533-8e7c-5a7a886886be None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:06.106155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:06.111059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022763f9-43bb-4100-bd05-3db15de7a890 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:06.111466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022763f9-43bb-4100-bd05-3db15de7a890 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:06.111567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022763f9-43bb-4100-bd05-3db15de7a890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:06.131390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-022763f9-43bb-4100-bd05-3db15de7a890 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:06.132618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-022763f9-43bb-4100-bd05-3db15de7a890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:06.140412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:06.292611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476fdbf-8397-4533-8e7c-5a7a886886be None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:06.306831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476fdbf-8397-4533-8e7c-5a7a886886be None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:06.312063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:06.312658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:06.312931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:06.313200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:06.321809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:06.323374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234f344-491c-48d8-93e2-1df3c938720b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:06.333292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 764/1516] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:07.139311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5e0-eec0-4397-83d8-21c37dafd525 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:07.139928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5e0-eec0-4397-83d8-21c37dafd525 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:07.139928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5e0-eec0-4397-83d8-21c37dafd525 None None] PATH_INFO: `/v3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:07.141189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 753/1517] 10.222.0.75 () {54 vars in 925 bytes} [Thu Aug 4 10:23:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:23:07.148061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be563-bc0e-4fdb-b6ce-246ffc248357 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:07.148470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be563-bc0e-4fdb-b6ce-246ffc248357 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:07.148611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be563-bc0e-4fdb-b6ce-246ffc248357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:07.206196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-668be563-bc0e-4fdb-b6ce-246ffc248357 None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:07.207939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-668be563-bc0e-4fdb-b6ce-246ffc248357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:07.224895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:07.278527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 765/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Aug 4 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:11.238172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e833-dbf6-41b2-8918-f333041ba9b2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:11.238703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e833-dbf6-41b2-8918-f333041ba9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:11.238703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e833-dbf6-41b2-8918-f333041ba9b2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:11.239985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 754/1519] 10.222.0.72 () {54 vars in 961 bytes} [Thu Aug 4 10:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:11.251555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bcf0a-b979-4f0d-bd11-bf92577d06cf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:11.251555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bcf0a-b979-4f0d-bd11-bf92577d06cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:11.251555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bcf0a-b979-4f0d-bd11-bf92577d06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:11.264992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-992bcf0a-b979-4f0d-bd11-bf92577d06cf None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:11.266184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-992bcf0a-b979-4f0d-bd11-bf92577d06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:11.275216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 766/1520] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Aug 4 10:23:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:11.282681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82e833-dbf6-41b2-8918-f333041ba9b2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:11.295800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82e833-dbf6-41b2-8918-f333041ba9b2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:11.300938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:11.301528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:11.301806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:11.302079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:11.310162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:11.311729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30476e2-144a-43f5-9931-9e4e553c070e service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:11.321642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 755/1521] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:17.535097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4ecf-204b-4094-8a6d-936b3ea4984c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:17.535622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4ecf-204b-4094-8a6d-936b3ea4984c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:17.535719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4ecf-204b-4094-8a6d-936b3ea4984c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:17.537059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:17.541876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5f21c-4600-4c2a-b2df-a36fe6a4d7a7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:17.542285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5f21c-4600-4c2a-b2df-a36fe6a4d7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:17.542429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5f21c-4600-4c2a-b2df-a36fe6a4d7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:17.562154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a5e5f21c-4600-4c2a-b2df-a36fe6a4d7a7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:17.563343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5e5f21c-4600-4c2a-b2df-a36fe6a4d7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:17.571735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:17.724368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e4ecf-204b-4094-8a6d-936b3ea4984c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:17.740297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e4ecf-204b-4094-8a6d-936b3ea4984c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:17.746508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:17.747498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:17.747775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:17.748079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:17.757352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:17.758915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9006a9-4267-408e-ae0b-c997796b2309 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:17.768110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 768/1524] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:18.588816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939888-a8f7-4180-8d88-8addcaf858a4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:18.589429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939888-a8f7-4180-8d88-8addcaf858a4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:18.589429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939888-a8f7-4180-8d88-8addcaf858a4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:18.590717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:18.595755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4ca9c-6712-415c-9c5c-b6d849de3145 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:18.596213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4ca9c-6712-415c-9c5c-b6d849de3145 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:18.596339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4ca9c-6712-415c-9c5c-b6d849de3145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:18.615714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8be4ca9c-6712-415c-9c5c-b6d849de3145 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:18.616939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8be4ca9c-6712-415c-9c5c-b6d849de3145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:18.624762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:18.774909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9939888-a8f7-4180-8d88-8addcaf858a4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:18.789340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9939888-a8f7-4180-8d88-8addcaf858a4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:18.794639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:18.795232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:18.795507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:18.795776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:18.804540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:18.806098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427b89c-8574-47e0-a970-869858ec784e service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:18.815139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 758/1527] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:22.810004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:22.810596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:22.810596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:22.811914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:22.817546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235975da-293d-4a14-8999-043610591aed None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:22.818062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235975da-293d-4a14-8999-043610591aed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:22.818160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235975da-293d-4a14-8999-043610591aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:22.837768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-235975da-293d-4a14-8999-043610591aed None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:22.838948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-235975da-293d-4a14-8999-043610591aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:22.846747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:22.918787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:22.920480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:22.996094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa934a-fde6-4d07-a818-8f0e131183a8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:22.999668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:23.000326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:23.000605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:23.000879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:23.009655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:23.011095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93a10-6ff3-4b27-89c6-1a931be24557 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:23.020993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 771/1530] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:25.042705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a110-cfca-4d81-905d-23e714ecb4a3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:25.043247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a110-cfca-4d81-905d-23e714ecb4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:25.043247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a110-cfca-4d81-905d-23e714ecb4a3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:25.044656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:25.050833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a772cb-272f-41f8-bfa0-50e1a1b119e5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:25.051236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a772cb-272f-41f8-bfa0-50e1a1b119e5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:25.051386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a772cb-272f-41f8-bfa0-50e1a1b119e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:25.071388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e3a772cb-272f-41f8-bfa0-50e1a1b119e5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:25.072608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e3a772cb-272f-41f8-bfa0-50e1a1b119e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:25.080395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:25.117135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714a110-cfca-4d81-905d-23e714ecb4a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:25.127891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714a110-cfca-4d81-905d-23e714ecb4a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:25.131593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:25.132231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:25.132518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:25.132796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:25.141780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:25.143205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c20298-a5af-41d6-bd95-f8f85b7a8d07 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:25.152966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 761/1533] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:28.727146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:28.727711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:28.727711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:28.729113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:28.734371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537eac-9bbd-4554-9e3d-80c25ccd276d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:28.734759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537eac-9bbd-4554-9e3d-80c25ccd276d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:28.734927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537eac-9bbd-4554-9e3d-80c25ccd276d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:28.742598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:28.755761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0a537eac-9bbd-4554-9e3d-80c25ccd276d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:28.756994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a537eac-9bbd-4554-9e3d-80c25ccd276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:28.764808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:29.069182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:29.070847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:29.134880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:29.148255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d4401-db09-492a-9e14-b2e2c36f2d54 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:29.151761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:29.152414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:29.152687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:29.152957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:29.161860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:29.163270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a61e8d-585c-46d5-91e5-ff22f23c5abc service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:29.172754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 774/1536] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:31.236697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:31.237239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:31.237239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:31.239117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:23:31.244360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425b9a-c8d2-486d-af8c-076a4e806e84 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:31.244798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425b9a-c8d2-486d-af8c-076a4e806e84 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:31.244910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425b9a-c8d2-486d-af8c-076a4e806e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:31.264665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-17425b9a-c8d2-486d-af8c-076a4e806e84 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:23:31.265856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-17425b9a-c8d2-486d-af8c-076a4e806e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:31.273612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:23:31.574291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:23:31.575996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:23:31.589439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce94f-44dc-46f8-adab-6ed297b2645e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:23:31.592936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:23:31.593535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:23:31.593818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:23:31.594094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:23:31.603397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:23:31.604876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95018d-9ab4-4492-b292-8da208ab321b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:23:31.614744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 764/1539] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:23:41.259927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=11 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:41.610057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:58.137998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.139084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.139084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.166127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.167327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.179685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 776/1540] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.188689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.189662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.190184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.216984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.218590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.232209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 765/1541] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.237945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.249461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8fc90-6a3a-4be6-a275-33e6e90f00c4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.261811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.262482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.262759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.263156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.264186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:58.264498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.266849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.284442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 777/1542] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:58.292624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.307620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225694-106d-4463-b218-e294ad7748b4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.321044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.328059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.329577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.330171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.331321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.333696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb6dfa-568e-4106-89a2-5e495d77e9e1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.394162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 766/1543] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:58.398808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.415310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b1291-c1bd-4df7-8ff6-8953c53b07c7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.420777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4c277-a3a6-464c-8698-9899815f8bd9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.421451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4c277-a3a6-464c-8698-9899815f8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.422014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4c277-a3a6-464c-8698-9899815f8bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.454635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1fe4c277-a3a6-464c-8698-9899815f8bd9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.461729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1fe4c277-a3a6-464c-8698-9899815f8bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.501011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.501664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.501943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.502216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.502900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.505505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.564108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 778/1544] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:58.570728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.573971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 767/1545] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:24:58.585335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.585983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.586397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ee2b-f7c3-460b-ae82-0588ee4ef37f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.586738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.597354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.597995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.598275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.598549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.599254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.600928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.619981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 779/1546] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:24:58.619981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.621512 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.623659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.643144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 768/1547] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:24:58.649821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.650557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802314a-e7a6-48bb-b6ee-6f6de7dbc361 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.664457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f294205-95c4-4a3e-872f-e9d5fc2c1e2e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.672018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.672018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.672018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.672018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.672689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.672689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2520636-d79a-481f-aeaf-7818c47b4ecd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.680672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.681631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.682151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.682649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.683598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:58.684151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.685537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:24:58.685537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:24:58.685537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:24:58.686857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.712160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 780/1548] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:24:58.712160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.712160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.712160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.715753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 769/1549] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:58.716923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.728295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab272-bceb-4e0f-bec3-546cb686ec87 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.741310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.742003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.742282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.742554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] PATH_INFO: `/v3/projects/9467bbafb0ea449dbc1c0b50fd0819e1/users/7fa64eabc9a746a68d4fbd72053a6aa6/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.743755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.744996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.762238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 781/1550] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.763057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:58.774590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9467bbafb0ea449dbc1c0b50fd0819e1, user_id=7fa64eabc9a746a68d4fbd72053a6aa6, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.777309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a11cd-d2d4-4be4-9740-5cc247e85f1d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.783791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1dc1-9c2e-456d-86be-cad06b2a2fc9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:58.789046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:58.789679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.790163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.790377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.791052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:58.792101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:24:58.792101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:24:58.792101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:24:58.793295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a830fc0-37ca-462e-86fe-d99acb8d6d27 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:58.817040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 770/1551] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:24:58 2022] PUT /identity/v3/projects/9467bbafb0ea449dbc1c0b50fd0819e1/users/7fa64eabc9a746a68d4fbd72053a6aa6/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:24:58.818484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae9ea1-90ff-4a42-b550-2cb9bd78ba4b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.819348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae9ea1-90ff-4a42-b550-2cb9bd78ba4b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.820487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae9ea1-90ff-4a42-b550-2cb9bd78ba4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.846468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 782/1552] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:58.847732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49103-245c-4389-b9fa-94428c6d73b3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.848050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49103-245c-4389-b9fa-94428c6d73b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.848324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49103-245c-4389-b9fa-94428c6d73b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.853923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9ae9ea1-90ff-4a42-b550-2cb9bd78ba4b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.855236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9ae9ea1-90ff-4a42-b550-2cb9bd78ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.868885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-00d49103-245c-4389-b9fa-94428c6d73b3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.870074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-00d49103-245c-4389-b9fa-94428c6d73b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.915265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 771/1553] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.916468 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a15b8-9d9a-44b7-a891-007f97a4a428 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.916756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a15b8-9d9a-44b7-a891-007f97a4a428 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.917033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a15b8-9d9a-44b7-a891-007f97a4a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.936425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 783/1554] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.938284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50e2e-c280-40d9-b6c5-ba4fb1b16f9d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.938588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50e2e-c280-40d9-b6c5-ba4fb1b16f9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.938920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50e2e-c280-40d9-b6c5-ba4fb1b16f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.944743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c2a15b8-9d9a-44b7-a891-007f97a4a428 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.946013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c2a15b8-9d9a-44b7-a891-007f97a4a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.953886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 772/1555] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.957770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31aea-ecf4-42cf-8cb8-807b18f60a16 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.957770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31aea-ecf4-42cf-8cb8-807b18f60a16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.957770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31aea-ecf4-42cf-8cb8-807b18f60a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:58.968592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-69f50e2e-c280-40d9-b6c5-ba4fb1b16f9d None None] MFA Rules not processed for user `7fa64eabc9a746a68d4fbd72053a6aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.969830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-69f50e2e-c280-40d9-b6c5-ba4fb1b16f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.974107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ab31aea-ecf4-42cf-8cb8-807b18f60a16 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:58.975335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ab31aea-ecf4-42cf-8cb8-807b18f60a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:58.977885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:58.984423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 773/1556] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:58.993567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:58.993851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:58.994151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.008748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:59.022548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.025349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.033445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 774/1557] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.034195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.044305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 784/1558] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.045569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f26f6e-12c2-4efd-83f4-93e96a5a4c05 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.045858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f26f6e-12c2-4efd-83f4-93e96a5a4c05 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.046133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f26f6e-12c2-4efd-83f4-93e96a5a4c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.050045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830be66-0841-4cdb-8e25-d3f189e4e776 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.054737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.055611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.055891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.056203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.056934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.058906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:24:59.058906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:24:59.058906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:24:59.060712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0afd62-f13f-463a-85ab-6221d3854d44 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.070270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b5f26f6e-12c2-4efd-83f4-93e96a5a4c05 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.071195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b5f26f6e-12c2-4efd-83f4-93e96a5a4c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.092027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 775/1559] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/users => generated 287 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.092295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 785/1560] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.095109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.095303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1af3e-fa65-4988-a310-e45390f96107 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.095467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.095629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1af3e-fa65-4988-a310-e45390f96107 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.095775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.095919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1af3e-fa65-4988-a310-e45390f96107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.118438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ff1af3e-fa65-4988-a310-e45390f96107 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.118664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.119646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ff1af3e-fa65-4988-a310-e45390f96107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.119768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.128662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 776/1561] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.128801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 786/1562] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.129272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.129737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447def2-0b58-48e3-8774-fd2069b23ae0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.129960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447def2-0b58-48e3-8774-fd2069b23ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.130239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447def2-0b58-48e3-8774-fd2069b23ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.138877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23256783-8b48-4f87-bab4-cd976847a0a2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.143540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.144180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.144464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.144740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.145433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.145737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.147327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8d38d-1023-4e0d-a001-d33a7e29c4f8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.148818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4447def2-0b58-48e3-8774-fd2069b23ae0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.149999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4447def2-0b58-48e3-8774-fd2069b23ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.159198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 787/1563] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.160507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.160791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.161065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 777/1564] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294efb6-8345-4dad-addd-ccc9c4a7a816 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294efb6-8345-4dad-addd-ccc9c4a7a816 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294efb6-8345-4dad-addd-ccc9c4a7a816 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 778/1565] 10.222.0.72 () {54 vars in 962 bytes} [Thu Aug 4 10:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.169119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.169682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.180719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.181907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.188433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.189600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.190304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 788/1566] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.191067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.197501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 779/1567] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.198291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.200549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd1ece-6111-469f-8339-f15884ae36e9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.205180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.205774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.206053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.206326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.207012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.207489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b7cc4-ad3e-4202-ba1a-6bd3e9d8dda8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.208639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.212088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.212686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.212971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.213249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.213944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.214252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.215830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.218606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 789/1568] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:24:59.219948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.228063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 780/1569] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.228947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.229132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd299-7fb6-4a1a-80b0-7e4f796df0b3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.233670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.234264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.234603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.235370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.236734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.237040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.238626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.246803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a8ba-823c-449d-868e-a579f9ac6390 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.249604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.250915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.251195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 790/1570] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.251324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.251491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.252005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.252416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.252734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.254386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69048b5e-c164-4ebf-a751-df8503cac05e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.268361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47def7-80a5-4a0c-b1d1-7b9056f6c5eb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.269779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 781/1571] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.271043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.271337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.271638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.280020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.280661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.280942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.282119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.282119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.283806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.310780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 791/1572] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.311598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.311916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:59.313677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:59.377396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0365-9d0d-4661-ac5a-db0cee92a0df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.382226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.382859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.383142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.383418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.384156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.384467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.384679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 782/1573] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:59.386180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.386337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.399261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 792/1574] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.399481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7173c7-33d0-42df-802e-1bc930c73108 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.399989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.410044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61ca7c-b53f-4af2-91df-d5d6c0847ee2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.411433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.412078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.412355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.412626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.413328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:24:59.413613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.414560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.415150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.415265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.415430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.415709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.416426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.418029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.429341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 783/1575] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:24:59.432928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 793/1576] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:24:59.433063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.433156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.444995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce388a9-6e9f-4999-a776-7549f626de63 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.445817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937877e-bf3d-419e-94d3-28a38c63f8ae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.449634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.450237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.450387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.450514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.450774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.450991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.451275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.451432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.451668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.452369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.453391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.454288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.474290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:59.485423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 794/1577] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.487470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.493498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 784/1578] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.494233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.527555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:59.552167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196daa3a-2d4b-4478-af03-bfdf6700379d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.556776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.557376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829893bc-67cf-4e74-89e7-1837684ad456 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.557523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.557680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.558048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.558581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.560540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.562145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.564172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.564172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.564172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.564529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.566432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.591044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 785/1579] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.591262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 795/1580] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.591366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.591464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.650486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39432-1de8-44db-96c7-9a53c7a32aa6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.655121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.655733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.656047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.656348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.657017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.659002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.661237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2baab-70b4-4aa0-a394-393211df7f58 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.685291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 796/1581] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.686147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.728616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.729553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.729838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.730133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.748307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f5584-f44c-4a2c-8ad1-626cc0f3ff7f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.753865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.754586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.754868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.755140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] PATH_INFO: `/v3/projects/9134c9d38f1f4b5b9a592a0669e1ad73/users/1bf691162d004bec8ce6894e59b05df8/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.767314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:59.770867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9134c9d38f1f4b5b9a592a0669e1ad73, user_id=1bf691162d004bec8ce6894e59b05df8, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.772284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:24:59.772284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:24:59.772284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:24:59.774604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.792834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 797/1582] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:24:59 2022] PUT /identity/v3/projects/9134c9d38f1f4b5b9a592a0669e1ad73/users/1bf691162d004bec8ce6894e59b05df8/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:24:59.793633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.807003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.808801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.813271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:59.819863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 786/1583] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:59.821048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.833140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1717b1b-6fa3-416f-a71c-b94ac2d37770 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.858984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96533-8705-4401-84c9-37a39c06286e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.864115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.864727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.865003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.865294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.865926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.868100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.895470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.896329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.896679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.897104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.900651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 798/1584] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.900815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.900915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.901015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.926836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 787/1585] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:24:59.927867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:59.938338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01bfe7-adc9-47ed-a016-9311371816f1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.970684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73dc99-5a0f-4ace-a606-09b4a8cbe118 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:59.976992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.977609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.977896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.978895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.979768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:59.982696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:59.997044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:59.997745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:59.998020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:59.998298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:59.999070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.000921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.010977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 799/1586] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:24:59 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:00.011163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 788/1587] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.011947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.012371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.021831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304d39-a4ff-4624-a717-a3ca62a26c0d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.026823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.027452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.027731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.028029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.028789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.029034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd406691-6200-40ea-a047-d7e063b231a2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.030773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90d55a-2e26-4c96-aff7-09e8d20abe2a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.034601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.035214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.035498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.036518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.036627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.044163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.058305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 789/1588] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:00.058305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.058305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.058305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.072413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 800/1589] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/users => generated 290 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:00.072413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.080857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39d92f-d6ef-4312-b8cc-3284e03783d6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.085374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.085972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.086252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.086451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] MFA Rules not processed for user `1bf691162d004bec8ce6894e59b05df8`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.086566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.087166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.087638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.089113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.112695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 801/1590] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:00.113545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.123323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f4cba-7c72-42ce-b141-1c04d19d9ed8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.128134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.128755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.129033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.129303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.129984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.131559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.141398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 802/1591] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.142485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.147888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.152775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e17c2-29f9-4773-a165-2fcd654600c4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.159282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 790/1592] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.160118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.165166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.165784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.166067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.166338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.167027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.168670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.175229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b303a38-ab4e-4e43-9e42-4e4e98600683 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.180211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.183858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 803/1593] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.184024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.184024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.184024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.184254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.184362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.184362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6f764-65f4-4a9c-b23d-2efe17700423 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.194210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c180c-d170-45db-a4f3-7ff3bbc2b9f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.197699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 791/1594] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.197699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65192-9742-4131-980a-fbf44586f767 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.197699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65192-9742-4131-980a-fbf44586f767 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.197699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65192-9742-4131-980a-fbf44586f767 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.198398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 792/1595] 10.222.0.72 () {54 vars in 912 bytes} [Thu Aug 4 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:00.198685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65192-9742-4131-980a-fbf44586f767 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.206280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.206949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.207217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.207488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.208212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.209802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.211653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65192-9742-4131-980a-fbf44586f767 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.212546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.216466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.217147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.217430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.217705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.218405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.221160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.226758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 804/1596] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.226758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.231392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 793/1597] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.232487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.237001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41bf9-90d1-44ad-a2a0-164ac323b5f2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.241622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.242300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.242867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.243221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.243900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.244347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3f99-d259-446f-bead-ad9f69593f26 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.245524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.249231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.250042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.250208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.250490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.251236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.252875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75233d9e-e513-4170-b7f4-3c74289abba8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.256071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 805/1598] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.256932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.266970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 794/1599] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.266970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed0b3a-44ff-44cc-a454-5959d311b05d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.266970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed0b3a-44ff-44cc-a454-5959d311b05d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.266970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed0b3a-44ff-44cc-a454-5959d311b05d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.267540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972aab0-003d-43a5-8f8c-64440fa0f296 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.267645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 795/1600] 10.222.0.72 () {54 vars in 962 bytes} [Thu Aug 4 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:00.268010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed0b3a-44ff-44cc-a454-5959d311b05d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.272138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.272823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.273107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.273380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] PATH_INFO: `/v3/projects/888ab157217942abaf98b99e8eb827de/users/0ac9aa3a8d7a4383b9211cbac185a325/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.280481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed0b3a-44ff-44cc-a454-5959d311b05d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.285657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.286238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.286515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.286787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.287467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.288860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=888ab157217942abaf98b99e8eb827de, user_id=0ac9aa3a8d7a4383b9211cbac185a325, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.289092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b951816-f75d-4c67-8861-bffa425b46c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.291586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.300560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 796/1601] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.301771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.302055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.302326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.308766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 806/1602] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/888ab157217942abaf98b99e8eb827de/users/0ac9aa3a8d7a4383b9211cbac185a325/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.309369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.323505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.324716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.369687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951c8e-cd7c-4314-8657-f71812c6a0cc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.374308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.374922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.375235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.375511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.376266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.377866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.391246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 807/1603] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.391246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.394092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 797/1604] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.395130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.400591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e18c8-1b10-4d1f-a913-bf1889991db0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.405963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac8039-2b87-4ede-93e3-f734a3bed8e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.406132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.407375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.407606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.407886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.408616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.410244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.411607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.412316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.412667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.413002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.413757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.415398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313db240-2fdd-403e-bc06-ac565eb91ca9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.423657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 808/1605] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.423657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.425609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 798/1606] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:00.426784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.427070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.427345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.434259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde701-d891-4fd5-824f-4b43c77bb752 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.438954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.439609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.439893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.440198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] PATH_INFO: `/v3/projects/e48cfdacb9c44bdd9fe7df95ee93f65b/users/532df9b1c22b4b0c9237bb0a36044c7b/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.447107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.448274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.455268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48cfdacb9c44bdd9fe7df95ee93f65b, user_id=532df9b1c22b4b0c9237bb0a36044c7b, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.457911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 799/1607] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.458056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ccba6-1cfc-43d4-9ce7-fb6919753823 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.458649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.468524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5a99-8662-4b64-952f-06017e467a9c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.473880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 809/1608] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/e48cfdacb9c44bdd9fe7df95ee93f65b/users/532df9b1c22b4b0c9237bb0a36044c7b/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.475284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.475578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.476008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.495151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.496542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.524904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.525576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.525862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.526140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] PATH_INFO: `/v3/projects/ae1e51ae816f4dc6b7dac8c2ddcf38bc/users/b680411b142642b7acc3585d2481f880/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.541452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1e51ae816f4dc6b7dac8c2ddcf38bc, user_id=b680411b142642b7acc3585d2481f880, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.544251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.555572 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 810/1609] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.556840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.562501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 800/1610] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/ae1e51ae816f4dc6b7dac8c2ddcf38bc/users/b680411b142642b7acc3585d2481f880/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.562501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.567889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc2b60-5a55-429c-a2be-3ddd9eeee15f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.572790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.588597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.619720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37c89-da3b-47fa-a223-b090a93bc60b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.623418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.624065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.624346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.624618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.626227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.626888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.627163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.627514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] PATH_INFO: `/v3/projects/35576cef915b41419fc74b0beea0f988/users/7213149649d249f2b2da3d6c7dac0a68/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.642348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35576cef915b41419fc74b0beea0f988, user_id=7213149649d249f2b2da3d6c7dac0a68, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.645229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.659539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 811/1611] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/35576cef915b41419fc74b0beea0f988/users/7213149649d249f2b2da3d6c7dac0a68/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.660406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.697098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.698676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.710026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 801/1612] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:00.710691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.721107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810060cd-5e2d-4375-ae94-c0d0210b8be1 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.722407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d87616-7f97-431c-ba56-692d9003d065 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.725962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.726720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.727072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.727417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] PATH_INFO: `/v3/projects/2a801dbffe56474d96c2abea31fd2fdc/users/84b748f30ebc4c5aa2af6ddcd86037cd/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.728976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.730204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.730558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.731789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] PATH_INFO: `/v3/projects/1b9bcf3200c74d839b62c8105e98d047/users/7941455732644199a5e1ca091021809d/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.743434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a801dbffe56474d96c2abea31fd2fdc, user_id=84b748f30ebc4c5aa2af6ddcd86037cd, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.746123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.751741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9bcf3200c74d839b62c8105e98d047, user_id=7941455732644199a5e1ca091021809d, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.754659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec45d3-fe80-48a5-b661-86a63554aa8a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.762666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 802/1613] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/2a801dbffe56474d96c2abea31fd2fdc/users/84b748f30ebc4c5aa2af6ddcd86037cd/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.763645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.774190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 812/1614] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:00 2022] PUT /identity/v3/projects/1b9bcf3200c74d839b62c8105e98d047/users/7941455732644199a5e1ca091021809d/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:00.776510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e872e-14ed-4174-a5ef-b9417bdcd5ef None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.776953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e872e-14ed-4174-a5ef-b9417bdcd5ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.777722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e872e-14ed-4174-a5ef-b9417bdcd5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.799920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d51e872e-14ed-4174-a5ef-b9417bdcd5ef None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.801154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d51e872e-14ed-4174-a5ef-b9417bdcd5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.845134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d95f15-88df-4df4-b53b-4331a46fdba7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.868136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.869186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 813/1615] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.870557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.870848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.871123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.894979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:00.896390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:00.904925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 814/1616] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:00.905750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:00.929645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.930268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.930589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.930862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.933883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:00.971254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb5db-60bf-4e1c-9882-e391beee6b81 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:00.975283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:00.975934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:00.976247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:00.976523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:00.988696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.990946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794971d7-db44-464d-bc2a-0a6c933ee4c7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:00.995350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:00.997017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de835e2-35d8-4585-99ed-c887a112deec service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.006840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 803/1617] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.008199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb05e6c-33b9-4610-848f-49c42f2e3762 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.008477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb05e6c-33b9-4610-848f-49c42f2e3762 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.008768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb05e6c-33b9-4610-848f-49c42f2e3762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.009008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 815/1618] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.011289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.011680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.012239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.030232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5cb05e6c-33b9-4610-848f-49c42f2e3762 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:01.032521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5cb05e6c-33b9-4610-848f-49c42f2e3762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:01.038312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:01.039889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:01.044636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 804/1619] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:01.048445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.049525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.049950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.058185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 816/1620] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:01.059367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.069153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:01.083717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013c40b-4eb5-45d2-88de-cc0ff5b36dc7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.088042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.088883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.089347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.089897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.103225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.105311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.116415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] MFA Rules not processed for user `7941455732644199a5e1ca091021809d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:01.117619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:01.123908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 817/1621] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.137526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.159070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cd45-c483-4635-902a-688cba11dbe9 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.169276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.171614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.172643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.173359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.196043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.200867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.230245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 818/1622] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.232638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.235398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 805/1623] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:01.236345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.251565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bfd27-be3e-4400-8769-888116feb4a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.252599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad0b2-e8f5-4e47-bbe0-38cd99a03417 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.258621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.260437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.260919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.261405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.263509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.265509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.268989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.269426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.279443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.282054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.292652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.294160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95cc33-f3f0-4877-82f0-6c5c2ff90059 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.298957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 806/1624] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.301546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.307861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 819/1625] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.309839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f67a9e-60a9-471a-9bab-393c01a6c396 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.310313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f67a9e-60a9-471a-9bab-393c01a6c396 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.310748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f67a9e-60a9-471a-9bab-393c01a6c396 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.314386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a961-1301-47e9-8b8f-b67c1fff595a service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.316745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 820/1626] 10.222.0.72 () {54 vars in 912 bytes} [Thu Aug 4 10:25:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:01.318085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.318371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.318695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.321079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.321703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.321983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.322310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.322979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.325039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.345097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:01.346760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:01.365376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 821/1627] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:01.374472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 807/1628] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:01.374923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.382250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.413825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d268b0-f8d9-42e3-ae6c-04726b3a6845 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.421233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:01.577480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969cc6-323f-4e78-a72f-6a909fef7762 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.586346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.587050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.587362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.587655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.600926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.602818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.603137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.603414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.605583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.612337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.660478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 822/1629] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:01 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:01.664196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.683005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cafb-8f75-4434-ae6c-28d19abe0b55 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.691506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.692562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.693249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.693765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.694705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.697176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.701559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.703009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.714161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 823/1630] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:01.718388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.722877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 808/1631] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:01.755625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017f1-9fdd-406b-bde5-f1d0e42df2b0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.769878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.770591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.770945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.771333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] PATH_INFO: `/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b/users/eedef8732f85485cac607c3a9e9503c8/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.793946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f869f9a5eff24f70b5b74a1ca1c51c0b, user_id=eedef8732f85485cac607c3a9e9503c8, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.798201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.818601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 824/1632] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:01 2022] PUT /identity/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b/users/eedef8732f85485cac607c3a9e9503c8/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:01.819355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:01.839152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8181-0dd9-4cb5-9bbd-f75cf94f7c22 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:01.947643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:01.948974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:01.949576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:01.950064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:01.951455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:01.954903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb7ae4-6492-471e-a2ad-bf641ca5227d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:01.987607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 809/1633] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:01.993600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:02.005913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb4b94-550b-48bc-8751-12122de8dee4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:02.011694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:02.012841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.012841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.012841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] PATH_INFO: `/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b/users/eedef8732f85485cac607c3a9e9503c8/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.014746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f869f9a5eff24f70b5b74a1ca1c51c0b, user_id=eedef8732f85485cac607c3a9e9503c8, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:02.017909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:02.040785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 825/1634] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:01 2022] PUT /identity/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b/users/eedef8732f85485cac607c3a9e9503c8/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:02.043425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.043803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.044289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.108999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] MFA Rules not processed for user `eedef8732f85485cac607c3a9e9503c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:02.113497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:02.173288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:02.230796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 810/1635] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:02.242908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:02.266167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:02.318445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8accc25-ebee-4ade-a5e9-f57251c6dfd5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:02.324157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:02.325231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.325631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.326077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.340535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:02.342166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:02.364282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 826/1636] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:02.545724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:02.562006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db80aa-c5a8-4851-bb6f-30ae10c8fc2d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:02.570758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:02.571391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.571652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.571925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.728435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:02.730380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb23a9-7834-44f4-a938-297882e0af3c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:02.769121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 811/1637] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:07.599830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:07.799304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ca30c-5913-4f48-b220-f22e48e89034 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:07.807529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:07.809142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:07.809613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:07.810270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] PATH_INFO: `/v3/users/1bf691162d004bec8ce6894e59b05df8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:07.814729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf691162d004bec8ce6894e59b05df8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:07.816075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:25:07.816075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: warnings.warn(msg) Aug 04 10:25:07.816075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: Aug 04 10:25:07.819592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:08.049971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf691162d004bec8ce6894e59b05df8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:08.056680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf691162d004bec8ce6894e59b05df8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:08.059413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf691162d004bec8ce6894e59b05df8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:08.095616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf691162d004bec8ce6894e59b05df8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:08.123856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf691162d004bec8ce6894e59b05df8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:08.172794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 827/1638] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:07 2022] DELETE /identity/v3/users/1bf691162d004bec8ce6894e59b05df8 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:08.177405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:08.178059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:08.178059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:08.333700 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:08.339281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:08.562692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 812/1639] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:08.635786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:08.637615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:08.776525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:08.827186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e680-a7bf-4a00-b00a-467c47150cd7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:08.836920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:08.838382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:08.838671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:08.838953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:08.864490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:08.865974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9f069-c6be-464c-87b8-2864d2779e10 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:08.888044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 828/1640] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:12.205763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:12.207166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:12.261508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cc2c2-f3af-49b4-87a0-09e120ef07d5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:12.273659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:12.274587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:12.275016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:12.275458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] PATH_INFO: `/v3/users/7fa64eabc9a746a68d4fbd72053a6aa6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:12.277211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa64eabc9a746a68d4fbd72053a6aa6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:12.283436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:12.414962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa64eabc9a746a68d4fbd72053a6aa6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:12.415860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa64eabc9a746a68d4fbd72053a6aa6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:12.417966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa64eabc9a746a68d4fbd72053a6aa6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:12.458143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa64eabc9a746a68d4fbd72053a6aa6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:12.459310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:12.470954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa64eabc9a746a68d4fbd72053a6aa6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:12.520440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 813/1641] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:12 2022] DELETE /identity/v3/users/7fa64eabc9a746a68d4fbd72053a6aa6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:12.524909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:12.528326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:12.530503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:12.569781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:12.573149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:12.746975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 829/1642] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:12.887604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:12.891756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:12.964669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:12.966446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:12.987274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb61fa-fe1d-4448-9aba-7678c44f7be2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:13.008554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:13.009214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.009544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.009818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] PATH_INFO: `/v3/projects/9134c9d38f1f4b5b9a592a0669e1ad73` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.015973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9134c9d38f1f4b5b9a592a0669e1ad73)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:13.024115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:13.027064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54b52-16dd-4d50-a1e1-f0b35eb6a594 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:13.037465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:13.038615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.040397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.040816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.065031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:13.066673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1259-777c-461f-8eff-ca138d53bd38 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:13.095024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 814/1643] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:13.157987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9134c9d38f1f4b5b9a592a0669e1ad73', 'request_id': 'req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:13.173770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9134c9d38f1f4b5b9a592a0669e1ad73', 'request_id': 'req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:13.188590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9134c9d38f1f4b5b9a592a0669e1ad73', 'request_id': 'req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:13.204721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] The token cache is being invalidate because project 9134c9d38f1f4b5b9a592a0669e1ad73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:13.205011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:13.205710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a9a3a89-0a90-4d63-8d7a-2dc820da025b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:13.207715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 830/1644] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:12 2022] DELETE /identity/v3/projects/9134c9d38f1f4b5b9a592a0669e1ad73 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:13.212295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.212785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.212876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.250782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:13.252044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:13.477215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 815/1645] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:13.477410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.477674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.477858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.513142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:13.517219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:13.548087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 831/1646] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:13.549277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:13.564573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ef37-286a-4651-bc70-33eb63af28ce None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:13.575053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:13.575464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.575750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.576051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.576768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:13.577072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:13.582642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:13.673114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 816/1647] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:13.677173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:13.687251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb9db2-7690-40e2-92e5-149655434ff3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:13.701115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:13.701762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.702045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.702312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.703024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:13.705147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:13.814822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:13.829272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 832/1648] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:13.835053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23a61-a068-4b8f-9e92-b828eaac4aec admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:13.965011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:13.965983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.966566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.966906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.967643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:13.971279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.007891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 817/1649] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:14.008815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.023457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d995c5-6d86-4527-ad09-9db052d6f7bf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:14.062888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:14.063445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.065805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.066100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.066815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:14.072976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.114623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 833/1650] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:14.115919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.125057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:14.153311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd6c5-b2dc-4085-9e2f-ce302b2eb981 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:14.183141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:14.186438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.187277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.189045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.191709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:14.195442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023db13-3fa6-4e05-b0d3-fb3c65ac3b43 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.263898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 818/1651] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:14.266542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.284913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3456-d661-44eb-8fce-3f33768298c0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:14.293970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:14.294825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.295225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.295649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] PATH_INFO: `/v3/projects/ca4f7b78b1d043ddaf467c6a4bc038ba/users/9d7cce7717de4b5fb7f95bd743917f41/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.324138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4f7b78b1d043ddaf467c6a4bc038ba, user_id=9d7cce7717de4b5fb7f95bd743917f41, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:14.328432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786b0e-03c5-4dc5-8c44-287b8d278659 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.341203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:14.349089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 834/1652] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:14 2022] PUT /identity/v3/projects/ca4f7b78b1d043ddaf467c6a4bc038ba/users/9d7cce7717de4b5fb7f95bd743917f41/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:14.359536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.359826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.360149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.425865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] MFA Rules not processed for user `9d7cce7717de4b5fb7f95bd743917f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:14.430145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.549955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2987-bbfd-494c-acac-b5e35aaa80e2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.550840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2987-bbfd-494c-acac-b5e35aaa80e2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.551249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2987-bbfd-494c-acac-b5e35aaa80e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.587887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 819/1653] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:14.593133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.625873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-78da2987-bbfd-494c-acac-b5e35aaa80e2 None None] MFA Rules not processed for user `84b748f30ebc4c5aa2af6ddcd86037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:14.627622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-78da2987-bbfd-494c-acac-b5e35aaa80e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.757966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d3f62-36bd-478b-b5b6-ddb1eba0c125 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:14.779530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:14.780209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.780494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.780773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.785909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:14.793834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 835/1654] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:14.804333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.805395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.805669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.805939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.853354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] MFA Rules not processed for user `84b748f30ebc4c5aa2af6ddcd86037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:14.854775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.872729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 836/1655] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:14.895289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 820/1656] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:14 2022] POST /identity/v3/projects => generated 369 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:14.902402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.910266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.912056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.928893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea9442-b179-44e5-bc76-17042d2d0272 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.074939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.075685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.076109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.076486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.080298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.082932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.107505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:15.116797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276087b-2cc4-4538-bb11-5eb55d4d324b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.136340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.137101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.137270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.137375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 837/1657] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:14 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:15.137546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.138656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.141231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.163012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248ee0-ce29-40b0-9923-3e72002bd9d2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.171697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.172952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.173519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.173976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] PATH_INFO: `/v3/projects/9467bbafb0ea449dbc1c0b50fd0819e1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.175678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9467bbafb0ea449dbc1c0b50fd0819e1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.178001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.281919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9467bbafb0ea449dbc1c0b50fd0819e1', 'request_id': 'req-f0e42bbb-3043-4074-b6e6-7117406541c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:15.294407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9467bbafb0ea449dbc1c0b50fd0819e1', 'request_id': 'req-f0e42bbb-3043-4074-b6e6-7117406541c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:15.301142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.303247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.316583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 821/1658] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:15.317883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.318316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9467bbafb0ea449dbc1c0b50fd0819e1', 'request_id': 'req-f0e42bbb-3043-4074-b6e6-7117406541c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:15.321347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:15.332106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad73311-6074-4363-a9ee-675a8466e62d service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.337013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] The token cache is being invalidate because project 9467bbafb0ea449dbc1c0b50fd0819e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:15.337385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:15.338238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f0e42bbb-3043-4074-b6e6-7117406541c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:15.340389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 838/1659] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:15 2022] DELETE /identity/v3/projects/9467bbafb0ea449dbc1c0b50fd0819e1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:15.360229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1f64b-cafe-4ee2-8ce0-2551adc1e6f4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.361378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1f64b-cafe-4ee2-8ce0-2551adc1e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.361823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1f64b-cafe-4ee2-8ce0-2551adc1e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.410414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-80e1f64b-cafe-4ee2-8ce0-2551adc1e6f4 None None] MFA Rules not processed for user `0ac9aa3a8d7a4383b9211cbac185a325`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:15.413280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80e1f64b-cafe-4ee2-8ce0-2551adc1e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.427457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.428214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.428501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.428778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.429584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.431212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd43439-e4b2-4524-ba72-5d50d77473e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.461077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 822/1660] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:15.462811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.465349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.466125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.521114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:15.523288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.587387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 823/1661] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:15.589483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.594205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.620089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eeac2-0eca-477f-bc18-ee319d19dfe9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.638469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.639166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.639478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.639754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] PATH_INFO: `/v3/projects/1692cadf3a0d40e2afd3c184f55de873/users/333d46dd565546d0a49f73e1c38259e2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.656721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 839/1662] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:15.657946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.658237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.658512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.673649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1692cadf3a0d40e2afd3c184f55de873, user_id=333d46dd565546d0a49f73e1c38259e2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.678022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.712956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:15.715266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.730215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 824/1663] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:15 2022] PUT /identity/v3/projects/1692cadf3a0d40e2afd3c184f55de873/users/333d46dd565546d0a49f73e1c38259e2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:15.730215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.731364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:15.902070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 840/1664] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:15.902965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.927790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d726-08d7-4b03-8265-81cbae76d741 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.932586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.933830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.934121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.934400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.936540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:15.940378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:15.959588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf62e07-0a36-4505-a4e3-3d483fe95f74 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:15.967928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:15.969027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.969522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.970005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.971710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 841/1665] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:15.974386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:15.978339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.005895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f96ef-3c19-410b-b446-2538ad6cb090 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.022316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.022964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.023244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.023517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.024249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:16.024789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.036574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.067171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 842/1666] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:16.068600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.078631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.103566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05fa2-0390-44a6-b603-7daa5d3c211b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.124798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.125639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.125996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.126359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] PATH_INFO: `/v3/users/7941455732644199a5e1ca091021809d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.128376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7941455732644199a5e1ca091021809d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.130408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.134285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.139100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.162943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 825/1667] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:16.163729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.191922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285f5f-695e-4e30-aa34-88a3ad2fe595 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.205043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.207545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.208110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.208939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] PATH_INFO: `/v3/projects/1692cadf3a0d40e2afd3c184f55de873/users/333d46dd565546d0a49f73e1c38259e2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.216279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1692cadf3a0d40e2afd3c184f55de873, user_id=333d46dd565546d0a49f73e1c38259e2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.221813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.262851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7941455732644199a5e1ca091021809d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:16.263577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7941455732644199a5e1ca091021809d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:16.264198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7941455732644199a5e1ca091021809d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:16.275365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.287406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7941455732644199a5e1ca091021809d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:16.297885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 826/1668] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:16 2022] PUT /identity/v3/projects/1692cadf3a0d40e2afd3c184f55de873/users/333d46dd565546d0a49f73e1c38259e2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:16.298903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.300236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8a120ad5-eb76-43f0-bc53-68d2c734ef80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7941455732644199a5e1ca091021809d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:16.308872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.319575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 843/1669] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:16 2022] DELETE /identity/v3/users/7941455732644199a5e1ca091021809d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:16.321237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408ba9-cc86-4aae-8859-7bc4e83104af None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.321525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408ba9-cc86-4aae-8859-7bc4e83104af None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.321798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408ba9-cc86-4aae-8859-7bc4e83104af None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.323368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 844/1670] 10.222.0.72 () {54 vars in 912 bytes} [Thu Aug 4 10:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:16.324510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1380d3-52ce-476d-86a2-89ea306a3c72 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.324788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1380d3-52ce-476d-86a2-89ea306a3c72 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.325129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1380d3-52ce-476d-86a2-89ea306a3c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.367724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8a1380d3-52ce-476d-86a2-89ea306a3c72 None None] MFA Rules not processed for user `333d46dd565546d0a49f73e1c38259e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:16.368910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8a1380d3-52ce-476d-86a2-89ea306a3c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.455776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5359aa3-1078-4b93-83b6-941bd85bafc3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.533361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.534158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.534380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.534648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.535335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.538462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28732abc-a30f-408e-b952-36b481f51ef8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.566919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 845/1671] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.568610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.568967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.570027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.599499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 827/1672] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/projects => generated 405 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:16.600791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.601082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.601356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.621031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:16.629272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.706570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:16.707813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.734809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.781827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 846/1673] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.782780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.786407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.825463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 828/1674] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.826372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.842056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b4d24-8a5d-4eff-acda-05559e47ced9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.850598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.851257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.851607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.851977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.852876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:16.855523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:16.921236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a877cf-4113-4f6e-9b22-86987cdd6f29 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.928189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.928830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.929114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.929388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.933666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 829/1675] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:16.934797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.940095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.956679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.976762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883568-475a-42a4-8c83-d069dc2537f5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.986511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.986730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.986975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.987652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.009776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.014775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.027299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 830/1676] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:17.028933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.039300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787530-905e-4dc6-a5f9-39fd1294eac6 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.044747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.045429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.045711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.045988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.063270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.064919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.075180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 831/1677] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:17.076094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.084270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.085734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.090146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362bc7-42fa-4ecf-a4d6-ab1c44159636 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.096338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.099065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.099359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.099629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.100909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.103341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.118740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 832/1678] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:17.125733 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.129987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 847/1679] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:17.137624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95b7f-f962-4a8c-b269-6b137ba3d025 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.144605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.145472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.145703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.145974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.146682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.148445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.168372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 833/1680] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:17.168969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.213270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb13917-1a97-45ba-916c-a84efded5efb service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.224606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.227089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:17.232185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.232882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.233157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.233436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] PATH_INFO: `/v3/projects/2eacf8c8f1374702948a3a4d9ee8586a/users/3dadde5b228f44f984df17b266c24534/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.263928 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ea9c7-3d31-466e-b422-976176e9139b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.266350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eacf8c8f1374702948a3a4d9ee8586a, user_id=3dadde5b228f44f984df17b266c24534, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.269081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a969677-10e0-40ca-bfcb-2d4798761f9d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.269690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.270688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.271200 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.271663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.275822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:17.316059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 848/1681] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:17 2022] PUT /identity/v3/projects/2eacf8c8f1374702948a3a4d9ee8586a/users/3dadde5b228f44f984df17b266c24534/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:17.320719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.320997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.321277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.393808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] MFA Rules not processed for user `3dadde5b228f44f984df17b266c24534`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:17.395041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:17.406603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.409275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.424757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 834/1682] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:17.575213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 849/1683] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:17.580358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.595220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bbefc-c107-4ced-b484-6ee350b157f7 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.737119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:17.763787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.766413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.766704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.766982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.771739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.785216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.927107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:17.929420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 835/1684] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:17 2022] POST /identity/v3/projects => generated 403 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:17.953600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098a888-4fd0-4eb7-87b9-542c5c9a3754 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.203507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.204478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.204934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.205217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.208080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:18.211888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:18.283901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 850/1685] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:17 2022] POST /identity/v3/users => generated 308 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:18.285266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:18.301688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8515-a30e-4edf-8cca-bfaf068997c2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.316129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.319338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.319801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.320114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.320846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:18.322465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:18.396776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:18.397343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 836/1686] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:18.408820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:18.416708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691a69-941e-4880-991f-94e56f219afd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.426647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.430215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.434493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.435240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] PATH_INFO: `/v3/projects/1623d5c3d779497eb200aa4e8c931189/users/76d4c93a51654ca7bc570989ad8d8283/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.477293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1623d5c3d779497eb200aa4e8c931189, user_id=76d4c93a51654ca7bc570989ad8d8283, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:18.480396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3aefc-97c2-418f-bf1d-e09712ff0f48 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:18.514738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 851/1687] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:18 2022] PUT /identity/v3/projects/1623d5c3d779497eb200aa4e8c931189/users/76d4c93a51654ca7bc570989ad8d8283/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:18.519411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:18.538376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2484a-ad6f-4937-b924-ab5c63cca69c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.561236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3869d-a9dc-43f2-8d45-718ba3e3a083 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.561526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3869d-a9dc-43f2-8d45-718ba3e3a083 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.561806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3869d-a9dc-43f2-8d45-718ba3e3a083 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.563806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 852/1688] 10.222.0.72 () {54 vars in 961 bytes} [Thu Aug 4 10:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:25:18.573878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.574765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.575225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.635412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:18.638985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:18.712325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.713379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.719101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.724225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.725808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:18.737712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b80406-a1f4-433b-925a-77f179e36e46 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:18.763061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 837/1689] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:18.765060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3410bf-d9b3-4473-a4c0-bf0b93df3c7a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.765400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3410bf-d9b3-4473-a4c0-bf0b93df3c7a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.765681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3410bf-d9b3-4473-a4c0-bf0b93df3c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.793869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 853/1690] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Aug 4 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:18.794978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:18.813444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5aca2a-70cc-4ae5-8c45-a5e0b30cf864 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.817180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b3410bf-d9b3-4473-a4c0-bf0b93df3c7a None None] MFA Rules not processed for user `84b748f30ebc4c5aa2af6ddcd86037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:18.825025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b3410bf-d9b3-4473-a4c0-bf0b93df3c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:18.828624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.829526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.829909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.830352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] PATH_INFO: `/v3/projects/1623d5c3d779497eb200aa4e8c931189/users/76d4c93a51654ca7bc570989ad8d8283/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.832863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1623d5c3d779497eb200aa4e8c931189, user_id=76d4c93a51654ca7bc570989ad8d8283, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:18.835837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:18.851636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 854/1691] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:18 2022] PUT /identity/v3/projects/1623d5c3d779497eb200aa4e8c931189/users/76d4c93a51654ca7bc570989ad8d8283/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:18.852706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:18.956920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962e6c-6505-4cb9-a212-bf0610cfcf37 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:18.960707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 838/1692] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:18.962185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.962466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.963178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:18.991864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:18.998452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:18.998969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:18.999479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:19.009194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:19.029558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:19.031974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:19.042020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] MFA Rules not processed for user `76d4c93a51654ca7bc570989ad8d8283`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:19.043208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:19.078571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 855/1693] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:19.133824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:19.145604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 839/1694] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:19.158524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:19.301858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61fd9c-9b22-415f-9dbe-f7d87deba41d service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:19.305613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:19.306776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:19.307059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:19.307334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:19.329677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:19.338089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:19.340078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08bdf3-ee61-4926-ac70-3d8faf0ab812 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:19.384284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 856/1695] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:19.604101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:19.752683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ae6b-7dce-44ad-9faa-0778bcfb9f4d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:19.762816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:19.763742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:19.764243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:19.765243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] PATH_INFO: `/v3/users/3dadde5b228f44f984df17b266c24534` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:19.766773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dadde5b228f44f984df17b266c24534)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:19.768966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:19.947169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dadde5b228f44f984df17b266c24534'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:19.950345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dadde5b228f44f984df17b266c24534'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:19.952718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dadde5b228f44f984df17b266c24534'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:19.996191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dadde5b228f44f984df17b266c24534'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.039000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dadde5b228f44f984df17b266c24534'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.088332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 840/1696] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:19 2022] DELETE /identity/v3/users/3dadde5b228f44f984df17b266c24534 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:20.092705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:20.093671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:20.094195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:20.166868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:20.168233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.358028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 857/1697] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:20.420815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:20.422571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:20.422701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.425454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.450380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a410b3c-76e1-4071-afff-97a64f0b9503 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:20.462526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:20.466904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:20.467206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:20.467490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] PATH_INFO: `/v3/projects/1b9bcf3200c74d839b62c8105e98d047` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:20.468912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9bcf3200c74d839b62c8105e98d047)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:20.469985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:25:20.469985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 04 10:25:20.469985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: Aug 04 10:25:20.477083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:20.620972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981c9f8-1337-4a3e-ae86-98c0e0beebb7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:20.649999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9bcf3200c74d839b62c8105e98d047', 'request_id': 'req-7445f870-cd79-40ec-8227-94d71dd20aa9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.678860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9bcf3200c74d839b62c8105e98d047', 'request_id': 'req-7445f870-cd79-40ec-8227-94d71dd20aa9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.733276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9bcf3200c74d839b62c8105e98d047', 'request_id': 'req-7445f870-cd79-40ec-8227-94d71dd20aa9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.742152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:20.767832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] The token cache is being invalidate because project 1b9bcf3200c74d839b62c8105e98d047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:20.768144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.770104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:20.779314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 841/1698] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:20 2022] DELETE /identity/v3/projects/1b9bcf3200c74d839b62c8105e98d047 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:20.784416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:20.785069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:20.785352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:20.785630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:20.789195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:20.791190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.792245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.942326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:20.943712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3313eaf-1315-4522-aca8-ef29380ec83d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:20.960752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 858/1699] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:21.013815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f870-cd79-40ec-8227-94d71dd20aa9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:21.035804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:21.037101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:21.038829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:21.039269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] PATH_INFO: `/v3/users/eedef8732f85485cac607c3a9e9503c8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:21.040890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eedef8732f85485cac607c3a9e9503c8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:21.061145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:21.216432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eedef8732f85485cac607c3a9e9503c8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:21.217967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eedef8732f85485cac607c3a9e9503c8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:21.219066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eedef8732f85485cac607c3a9e9503c8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:21.269611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eedef8732f85485cac607c3a9e9503c8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:21.289249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b4a2f2a-8bb0-4a90-a8c5-01f7ccf14ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eedef8732f85485cac607c3a9e9503c8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:21.324562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 842/1700] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:20 2022] DELETE /identity/v3/users/eedef8732f85485cac607c3a9e9503c8 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:21.419826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:21.422386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:21.422386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:21.442328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:21.513004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] MFA Rules not processed for user `532df9b1c22b4b0c9237bb0a36044c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:21.514251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:21.801216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 859/1701] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:21.812521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03e9b-cba4-4491-baa1-1f3f7c4f4e08 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:21.813146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03e9b-cba4-4491-baa1-1f3f7c4f4e08 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:21.813227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03e9b-cba4-4491-baa1-1f3f7c4f4e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:21.863380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-62a03e9b-cba4-4491-baa1-1f3f7c4f4e08 None None] MFA Rules not processed for user `532df9b1c22b4b0c9237bb0a36044c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:21.865416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-62a03e9b-cba4-4491-baa1-1f3f7c4f4e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:21.875548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 843/1702] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:21.892301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:21.894053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.153393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef76c89-b9d3-437a-b56e-42e603c68b34 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:22.176791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:22.177566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.177862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.178152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.203233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:22.204708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:22.215485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b2e5-66b2-416e-84a2-a8a24c1ad745 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.216401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b2e5-66b2-416e-84a2-a8a24c1ad745 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.216974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b2e5-66b2-416e-84a2-a8a24c1ad745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.253937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 860/1703] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:22.276319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b67b2e5-66b2-416e-84a2-a8a24c1ad745 None None] MFA Rules not processed for user `0ac9aa3a8d7a4383b9211cbac185a325`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:22.279834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b67b2e5-66b2-416e-84a2-a8a24c1ad745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.287451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:22.301190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.599528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36c2a-4c53-4911-81e1-f807db654876 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:22.638735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:22.638735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.639078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.639362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] PATH_INFO: `/v3/projects/2eacf8c8f1374702948a3a4d9ee8586a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.642373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eacf8c8f1374702948a3a4d9ee8586a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:22.646159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:22.654625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 844/1704] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:22.663291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.663643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.663846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.770211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:22.801759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:22.824121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] MFA Rules not processed for user `7213149649d249f2b2da3d6c7dac0a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:22.829825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.889314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eacf8c8f1374702948a3a4d9ee8586a', 'request_id': 'req-8c94b1e6-07da-481b-b1e4-07740d5e3da5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:22.919216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eacf8c8f1374702948a3a4d9ee8586a', 'request_id': 'req-8c94b1e6-07da-481b-b1e4-07740d5e3da5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:22.970107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eacf8c8f1374702948a3a4d9ee8586a', 'request_id': 'req-8c94b1e6-07da-481b-b1e4-07740d5e3da5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:22.995330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] The token cache is being invalidate because project 2eacf8c8f1374702948a3a4d9ee8586a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:22.995636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:22.997764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c94b1e6-07da-481b-b1e4-07740d5e3da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.010198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 861/1705] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:22 2022] DELETE /identity/v3/projects/2eacf8c8f1374702948a3a4d9ee8586a => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:23.010794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:23.011092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:23.023785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:23.117482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 845/1706] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:23.117482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:23.129369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.142327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] MFA Rules not processed for user `b680411b142642b7acc3585d2481f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:23.143567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.305994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4b68c-1683-4915-86dd-657a5b6e2706 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:23.318899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:23.319568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:23.319843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:23.322412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] PATH_INFO: `/v3/users/76d4c93a51654ca7bc570989ad8d8283` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:23.324100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d4c93a51654ca7bc570989ad8d8283)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:23.344860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:23.357427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 862/1707] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:23.358308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:23.363869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.399706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73106a79-bad4-4729-9101-2a5e7b856cc6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:23.419393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:23.421145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:23.422665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:23.423229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] PATH_INFO: `/v3/users/9d7cce7717de4b5fb7f95bd743917f41` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:23.425264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7cce7717de4b5fb7f95bd743917f41)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:23.429621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:23.481141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d4c93a51654ca7bc570989ad8d8283'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.481788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d4c93a51654ca7bc570989ad8d8283'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.482902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d4c93a51654ca7bc570989ad8d8283'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.514389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d4c93a51654ca7bc570989ad8d8283'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.515609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7cce7717de4b5fb7f95bd743917f41'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.517164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7cce7717de4b5fb7f95bd743917f41'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.518738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7cce7717de4b5fb7f95bd743917f41'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.536322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7cce7717de4b5fb7f95bd743917f41'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.545341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cc69d44-767c-4074-8b20-89c87b4fdf89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7cce7717de4b5fb7f95bd743917f41'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.545568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dcaabb-209d-4b54-bc8a-14c4ab754ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d4c93a51654ca7bc570989ad8d8283'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:23.572026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 863/1708] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:23 2022] DELETE /identity/v3/users/9d7cce7717de4b5fb7f95bd743917f41 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:23.575102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:23.575634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:23.575922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 846/1709] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:23 2022] DELETE /identity/v3/users/76d4c93a51654ca7bc570989ad8d8283 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:23.577342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:23.577550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:23.577760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:23.577856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:23.612278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:23.668281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:23.669555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.701435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:23.703824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.709151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:23.805704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 847/1710] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:23.871307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:23.873771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.878734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 864/1711] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:24.000577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad976e-2890-491e-b62f-ac778af6c4d5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:24.006540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:24.007507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.008560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.008977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.026185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:24.028203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:24.061915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 848/1712] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:24.112830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:24.115461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:24.309466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:24.311151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:24.332794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aef64-c685-4cab-864d-2ac9845745b4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:24.346281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:24.347264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.347789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.348828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] PATH_INFO: `/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.352785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f869f9a5eff24f70b5b74a1ca1c51c0b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:24.355153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:24.364930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801666-5493-440a-bc48-b5a190e3adb7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:24.375346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:24.376005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.376286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.376695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.397084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:24.399989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d184374-cea3-480f-aacc-6cb4556a5d39 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:24.424714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 865/1713] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:25:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:25:24.471579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f869f9a5eff24f70b5b74a1ca1c51c0b', 'request_id': 'req-c733a1b7-ccda-4966-a10d-b86750795261'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:24.478627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f869f9a5eff24f70b5b74a1ca1c51c0b', 'request_id': 'req-c733a1b7-ccda-4966-a10d-b86750795261'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:24.490080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f869f9a5eff24f70b5b74a1ca1c51c0b', 'request_id': 'req-c733a1b7-ccda-4966-a10d-b86750795261'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:24.510762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] The token cache is being invalidate because project f869f9a5eff24f70b5b74a1ca1c51c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:24.511545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:24.512525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c733a1b7-ccda-4966-a10d-b86750795261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:24.520832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 849/1714] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:24 2022] DELETE /identity/v3/projects/f869f9a5eff24f70b5b74a1ca1c51c0b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:24.525964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.526253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.526524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.569198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:24.570426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:24.692082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:24.733352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 866/1715] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:24.735357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.735644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.735926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.774127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:24.775401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:24.808556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 850/1716] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:24.810984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:24.833542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12973e-c062-411f-bd4e-71b46d161829 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:24.847518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:24.849456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.849733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.850008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.850865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:24.851199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:24.861280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:24.921370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 867/1717] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:24.923103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:24.938076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba07a1b-7b7d-43c4-b885-67829844214d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:24.974266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:24.975057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:24.975440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:24.976467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:24.978940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:24.997157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.105104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 851/1718] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:24 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:25.106712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.126276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f39b9-824a-4f54-b030-8809d7ef22ff admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.313706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.315429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.316918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.318456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.319766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.326151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.384993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 868/1719] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:25 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:25.388249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.405335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630edc49-35e2-4870-a6ef-e629f627f0b0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.420782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.422644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.432202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.433843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.436305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.451036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.481872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 852/1720] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:25.486405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.494219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.498848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b5404-6dce-44f2-a5a0-0c7c2f996b9b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.500404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:25.527439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.528110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.528390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.528662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.529362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.538615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.547808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212982-d308-478d-97d6-c7eb1ca4c6aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.566063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.566652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.566938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.567214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] PATH_INFO: `/v3/projects/ca4f7b78b1d043ddaf467c6a4bc038ba` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.575541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4f7b78b1d043ddaf467c6a4bc038ba)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.585812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.588452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 869/1721] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:25.589454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.594917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:25.642590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93d3b-258d-4825-8bdb-8fefd8dfdcc7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.661636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.662359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.662634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.662904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] PATH_INFO: `/v3/projects/c77ad00b3a0645daa530ed18d919ea0d/users/7cc0ae25dc75435988fdc25f4b09921b/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.759671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77ad00b3a0645daa530ed18d919ea0d, user_id=7cc0ae25dc75435988fdc25f4b09921b, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.762744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bc285-8fdf-4af5-9bf5-d3d5dc311fc6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.821189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 870/1722] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:25 2022] PUT /identity/v3/projects/c77ad00b3a0645daa530ed18d919ea0d/users/7cc0ae25dc75435988fdc25f4b09921b/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:25.821383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4f7b78b1d043ddaf467c6a4bc038ba', 'request_id': 'req-ef9ed74e-129e-407d-8716-3ed88d25d992'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:25.821484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb903d-fedb-4ca1-aab9-da5179714ff9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.821484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb903d-fedb-4ca1-aab9-da5179714ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.821484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb903d-fedb-4ca1-aab9-da5179714ff9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.825939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 871/1723] 10.222.0.72 () {54 vars in 912 bytes} [Thu Aug 4 10:25:25 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:25.833117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.833962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.835871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.856251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4f7b78b1d043ddaf467c6a4bc038ba', 'request_id': 'req-ef9ed74e-129e-407d-8716-3ed88d25d992'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:25.880919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4f7b78b1d043ddaf467c6a4bc038ba', 'request_id': 'req-ef9ed74e-129e-407d-8716-3ed88d25d992'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:25.902068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] The token cache is being invalidate because project ca4f7b78b1d043ddaf467c6a4bc038ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:25.905153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:25.905153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef9ed74e-129e-407d-8716-3ed88d25d992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:25.911192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 853/1724] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:25 2022] DELETE /identity/v3/projects/ca4f7b78b1d043ddaf467c6a4bc038ba => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:25.913294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.913472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.913811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.990486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] MFA Rules not processed for user `7cc0ae25dc75435988fdc25f4b09921b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:25.991718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:26.019868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:26.021131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:26.254403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 872/1725] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:26.257478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:26.259543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 854/1726] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:26.259656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:26.267156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:26.274559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:26.365220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:26.574131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8483-ca51-4a01-addb-4cbd3090b9d2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:26.578900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99f723-4ca0-4b6f-a23b-700819357614 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:26.594705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:26.595555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:26.595832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:26.596165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] PATH_INFO: `/v3/projects/1623d5c3d779497eb200aa4e8c931189` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:26.600004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:26.600828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:26.601042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:26.601323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] PATH_INFO: `/v3/users/333d46dd565546d0a49f73e1c38259e2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:26.604044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333d46dd565546d0a49f73e1c38259e2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:26.608892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:26.610956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1623d5c3d779497eb200aa4e8c931189)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:26.616369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:26.755414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333d46dd565546d0a49f73e1c38259e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.758027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333d46dd565546d0a49f73e1c38259e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.759365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333d46dd565546d0a49f73e1c38259e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.783580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:26.799732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333d46dd565546d0a49f73e1c38259e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.806657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1623d5c3d779497eb200aa4e8c931189', 'request_id': 'req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.829597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1623d5c3d779497eb200aa4e8c931189', 'request_id': 'req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.837735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333d46dd565546d0a49f73e1c38259e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.859344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1623d5c3d779497eb200aa4e8c931189', 'request_id': 'req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.881479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 873/1727] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:26 2022] DELETE /identity/v3/users/333d46dd565546d0a49f73e1c38259e2 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:26.882340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:26.891271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:26.909794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] The token cache is being invalidate because project 1623d5c3d779497eb200aa4e8c931189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:26.915509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.921092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:26.925412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 855/1728] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:26 2022] DELETE /identity/v3/projects/1623d5c3d779497eb200aa4e8c931189 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:26.930290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:26.933879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.263884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df385-1858-4f7a-ab19-3d038aa1e069 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:27.277562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:27.278782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.279397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.279821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.280814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:27.283751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7e7ca-040a-4b83-8b07-2dd1c2c745e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:27.313157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb76d-afc6-4b0b-a20d-f0f18341ad00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:27.366823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 874/1729] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:26 2022] POST /identity/v3/projects => generated 363 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:27.373255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.373563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.373846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.408138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:27.411663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.465835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:27.567787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:27.569361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.569645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.569924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.581860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.601328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 875/1730] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:27.602206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:27.629403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.813012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52007d0b-4a16-4278-83e3-937f5eb734ec None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:27.818269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:27.819350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.819907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.820672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.825558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.842782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:27.845146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa0acd-fc72-46a7-a61f-3005a771829e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:27.855714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:27.859586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 876/1731] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:27.863264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:27.870883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6bc97-2c85-4301-8d69-d2665b0a778b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.871601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6bc97-2c85-4301-8d69-d2665b0a778b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.872081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6bc97-2c85-4301-8d69-d2665b0a778b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.908498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 856/1732] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 981 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:27.909380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:27.909703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f3e6bc97-2c85-4301-8d69-d2665b0a778b None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:27.911298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f3e6bc97-2c85-4301-8d69-d2665b0a778b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.925999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.928184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 877/1733] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Aug 4 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:27.929759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:27.930182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:27.930626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:27.965695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:27.966325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.986609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 878/1734] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:27.988172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:27.989819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:27.990823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3de516-f5ac-4c6b-a5c7-1f9a5e0cc7e4 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.008720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784499f-2a71-41ad-82f7-ff641ec4fb95 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.013962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.015081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.016108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.017090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.021527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.022204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.022515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.022764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.023449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.025417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.059318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.060799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.061268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:28.074646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 879/1735] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:28.079383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.104437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68453cf5-6c9b-4a31-9b36-6e1df6336318 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.110450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.112104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.112614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.113036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.113891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:28.114364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.116283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.124384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 857/1736] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:27 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:28.127206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.163726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 880/1737] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:28.174136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.193843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd460bf-98ee-473b-8131-36afa6905b44 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.210640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.211282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.211562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.211833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.212628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.214291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.253660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 881/1738] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:28.254738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.288649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9387ab-c154-41ff-8e44-4e61e1f9057e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.312761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.313423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.313830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.313985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.314675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.316626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.392095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 882/1739] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:28 2022] POST /identity/v3/projects => generated 405 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:28.392993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.426440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e9b4-1c37-4325-8de2-f1fd246e9ba9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.497172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:28.502816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52660f7d-421b-4abe-ad13-e1baa285d966 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.518962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.521113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.521113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.521387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] PATH_INFO: `/v3/projects/0075a7ce29994737809711013e52c8b4/users/5cabe98f021e42e2855d27ce2dd7f8c2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.559061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0075a7ce29994737809711013e52c8b4, user_id=5cabe98f021e42e2855d27ce2dd7f8c2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.564863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.609732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 883/1740] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:28 2022] PUT /identity/v3/projects/0075a7ce29994737809711013e52c8b4/users/5cabe98f021e42e2855d27ce2dd7f8c2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:28.610771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.686065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.687036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.687434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.688788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.696561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:28.716743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9b93-5aa5-43c1-9f66-0129b8bf286f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.747249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.747972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.748262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.748530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.750046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.755782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.800085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 884/1741] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:28 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:28.800085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.826223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db01d7-e266-4c94-9d4a-0eb9b2db2cea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.870312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.871033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.871317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.871594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.878298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.886762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.892965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.894954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:28.915989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 885/1742] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:28.916865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.944573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 858/1743] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:28.945971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.968488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f142-3b10-49c8-804d-884520f04b11 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:28.986624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:28.987377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.987587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.987864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.988731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:28.990352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:29.004511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 886/1744] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:29.008941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:29.010182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcf53b-0cb6-4fd8-ae7f-a19c2ef6f9e6 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:29.026620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:29.028858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2920d2-64fc-4729-9316-ac765ac0b313 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:29.035186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.035533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.035852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] PATH_INFO: `/v3/projects/0075a7ce29994737809711013e52c8b4/users/5cabe98f021e42e2855d27ce2dd7f8c2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.056255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0075a7ce29994737809711013e52c8b4, user_id=5cabe98f021e42e2855d27ce2dd7f8c2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:29.064464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:29.066609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:29.073788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.073788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.075184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.076263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:29.078550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a49da-6b95-450d-80bb-8f22733f3737 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:29.106047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 859/1745] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:28 2022] PUT /identity/v3/projects/0075a7ce29994737809711013e52c8b4/users/5cabe98f021e42e2855d27ce2dd7f8c2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:29.112569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 887/1746] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:29.117559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:29.133981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.134382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.134813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.185574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] MFA Rules not processed for user `5cabe98f021e42e2855d27ce2dd7f8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:29.187537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:29.207523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:29.365850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 888/1747] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:29.378458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:29.381013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:29.403059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b385d9-1bd6-46f6-a75d-ec86e7cbbb1c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:29.425012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:29.425719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.426015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.426305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] PATH_INFO: `/v3/projects/71cd0200d71c477f8953d99a3500a588/users/ca09fbd6a3b141428edc2473e343b78d/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.461217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:29.491547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cd0200d71c477f8953d99a3500a588, user_id=ca09fbd6a3b141428edc2473e343b78d, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:29.505568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a5e1-f542-4086-afef-46e3fbaad242 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:29.551602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 860/1748] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:29 2022] PUT /identity/v3/projects/71cd0200d71c477f8953d99a3500a588/users/ca09fbd6a3b141428edc2473e343b78d/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:29.556782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288896-3427-4263-be80-e226c8c02836 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.557074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288896-3427-4263-be80-e226c8c02836 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.557355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288896-3427-4263-be80-e226c8c02836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.570828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5297c-dd07-4421-b051-9011226167d0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:29.631261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-66288896-3427-4263-be80-e226c8c02836 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:29.638677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-66288896-3427-4263-be80-e226c8c02836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:29.722248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:29.725104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.734330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.734787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.848118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 861/1749] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:29.982032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:29.986732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:30.046961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 889/1750] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:30.140597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66288896-3427-4263-be80-e226c8c02836 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:30.173849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:30.176020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:30.221653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff3e77-fe26-4148-a6e0-6c6cfb86a51e service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:30.241885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:30.242560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:30.242839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:30.246824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] PATH_INFO: `/v3/projects/1692cadf3a0d40e2afd3c184f55de873` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:30.252345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1692cadf3a0d40e2afd3c184f55de873)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:30.254400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:30.305559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66288896-3427-4263-be80-e226c8c02836 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:30.329629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:30.350619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:30.350960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:30.351255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:30.402236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:30.409772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064a469-fc3a-4fa0-bb92-f4d451e3848c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:30.442724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 862/1751] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:30.448405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1692cadf3a0d40e2afd3c184f55de873', 'request_id': 'req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:30.469552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1692cadf3a0d40e2afd3c184f55de873', 'request_id': 'req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:30.486440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1692cadf3a0d40e2afd3c184f55de873', 'request_id': 'req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:30.533366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] The token cache is being invalidate because project 1692cadf3a0d40e2afd3c184f55de873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:30.538121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:30.541797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcbbc6b7-41cb-49e6-8b8e-7741b2939b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:30.548496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 890/1752] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:30 2022] DELETE /identity/v3/projects/1692cadf3a0d40e2afd3c184f55de873 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:30.561166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:30.561679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:30.561679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:30.625407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:30.626779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:30.722521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:30.845024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:30.845341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:30.845695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:30.847228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 863/1753] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:30.895575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:30.897691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:30.933860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 891/1754] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:30.939202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:30.967066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf83b78-5bdf-4148-8376-317dc72b4951 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.001900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.002655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.003041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.004896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.005756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:31.011943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.013806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.046096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 864/1755] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:31.050074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.058696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:31.067079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169dde3-1731-40a3-850e-d308e0938aa8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.097640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.098741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.099477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.100079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.100987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.114613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.165695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.169665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:31.192788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 892/1756] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:31 2022] POST /identity/v3/projects => generated 393 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:31.199279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.208116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0a78d-f9fc-4827-9bb1-9d0a5366612c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.368897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f1e6-2523-4e65-8e6b-bb8475dfbac1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.377762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.378631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.378920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.379194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.381127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.383449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.420601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 893/1757] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:31 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:31.422650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.441477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2eb8d-424b-4ba8-bc70-3e09146938c1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.451979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.453217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.453630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.454057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.454994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.457496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.485413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 894/1758] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:31.487863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.489321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.489452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.489984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.490742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] PATH_INFO: `/v3/users/7cc0ae25dc75435988fdc25f4b09921b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.496756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc0ae25dc75435988fdc25f4b09921b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.507395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef239d-14d0-44d5-8f14-a57a1abab5c4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.507504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.514471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.516282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.516627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.516964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.517734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.520582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.548067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 895/1759] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:31.549085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.574836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a36ac2-6dd6-47fb-ad69-c7f397dcca25 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.603582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:31.607118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.609927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.612796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] PATH_INFO: `/v3/projects/a65361289774406b8bf06eaba0ae069c/users/8a82c36d7946405bad219e9379296861/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.677111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65361289774406b8bf06eaba0ae069c, user_id=8a82c36d7946405bad219e9379296861, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:31.684234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ceb6-275c-45c2-aab4-8195e1945953 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:31.685212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc0ae25dc75435988fdc25f4b09921b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:31.695023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc0ae25dc75435988fdc25f4b09921b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:31.695719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc0ae25dc75435988fdc25f4b09921b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:31.719967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 896/1760] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:31 2022] PUT /identity/v3/projects/a65361289774406b8bf06eaba0ae069c/users/8a82c36d7946405bad219e9379296861/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:31.721813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5d25-021e-4d0b-8421-df4d2f493d5d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.722264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5d25-021e-4d0b-8421-df4d2f493d5d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.722798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5d25-021e-4d0b-8421-df4d2f493d5d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.724731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 897/1761] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:31.746048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc0ae25dc75435988fdc25f4b09921b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:31.760888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f74742-7080-41ab-a144-b89e37419065 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.762839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f74742-7080-41ab-a144-b89e37419065 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.763904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f74742-7080-41ab-a144-b89e37419065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.780868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-14254e26-3f8d-412b-88cf-1c64804bfba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc0ae25dc75435988fdc25f4b09921b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:31.834473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 865/1762] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:31 2022] DELETE /identity/v3/users/7cc0ae25dc75435988fdc25f4b09921b => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:31.838201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.839513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.839810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.855630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66f74742-7080-41ab-a144-b89e37419065 None None] MFA Rules not processed for user `8a82c36d7946405bad219e9379296861`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:31.857185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66f74742-7080-41ab-a144-b89e37419065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:31.911885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:31.916219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.091753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:32.096794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 898/1763] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:32.097358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.097464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.097708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.143721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 866/1764] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:32.145823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:32.146442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:32.147694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.150051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.181895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:32.182102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8098d-6800-4808-862d-735d7e437209 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:32.197751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:32.197751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.197751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.198091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.199845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:32.204490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:32.319921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 867/1765] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:32 2022] POST /identity/v3/projects => generated 393 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:32.322156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:32.426157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 899/1766] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:32.428038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:32.434760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.710768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49314ed-e763-47aa-b816-043ff522f968 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:32.732196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:32.734905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.736773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.737534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.743726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:32.750904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:32.837914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b85d-81f6-49b3-afde-02a931d4b6d6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:32.869205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 868/1767] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:32 2022] POST /identity/v3/users => generated 303 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:32.873344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:32.881492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:33.020510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.026503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.026798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.031365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.079913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.081823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.103853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aaf50-1c45-4f7e-818d-691c35a64621 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.113367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.114030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.114320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.114599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.122818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 900/1768] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:33.126338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:33.129233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.131027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.182592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 869/1769] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:33.193314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00e703-fffb-4dfc-9593-186fdb63bec6 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.212049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.212840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.213113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.213601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.232665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.237834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.292579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 901/1770] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:33.298747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:33.319470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05869ce7-3bc1-4327-9a99-0b923a20c148 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.351862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.357463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.358099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.360933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] PATH_INFO: `/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638/users/c776366a686641a08789a3693fd6b92d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.462617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b5aae0f9f04a50b3b923d9b32e0638, user_id=c776366a686641a08789a3693fd6b92d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.480355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.539246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 870/1771] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:33 2022] PUT /identity/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638/users/c776366a686641a08789a3693fd6b92d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:33.545004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:33.570650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad46ec8-6b21-4e38-a7db-e8fc0d9e3a37 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.758014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:33.776283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.778840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.779483 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.780927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.783559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.786929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee230170-2e5d-4d65-83b6-50d7e2d3d2f8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.816176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 902/1772] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:33.819396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:33.836590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c152648-3ae6-4028-9def-490d2ff6010b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.850213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.852860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.853150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.853433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] PATH_INFO: `/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638/users/c776366a686641a08789a3693fd6b92d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.867159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b5aae0f9f04a50b3b923d9b32e0638, user_id=c776366a686641a08789a3693fd6b92d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.869459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.895653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:33.907322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 871/1773] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:33 2022] PUT /identity/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638/users/c776366a686641a08789a3693fd6b92d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:33.918671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.919611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.920038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.036533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] MFA Rules not processed for user `c776366a686641a08789a3693fd6b92d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:34.042648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.241253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 903/1774] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:34.258201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:34.260848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.442552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3340ac-91e8-4125-b744-67688b33bfca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:34.451824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:34.455074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.455395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.455642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.485476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:34.487047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc98d-eff3-4ebf-8ef7-927ebc2a287c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:34.525433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 872/1775] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:34.633983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:34.636191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.666892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe7b7-b60a-46b1-9919-da7230087b6c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:34.677873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:34.680916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.681261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.681546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.710145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c52c-203c-4476-b521-1fdceab3cdd3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.710731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c52c-203c-4476-b521-1fdceab3cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.711206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c52c-203c-4476-b521-1fdceab3cdd3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.713491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 873/1776] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:25:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:34.903587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:34.906341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:34.930023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 904/1777] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:35.641248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:35.641869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:35.641869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:35.692950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:35.694198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:35.871316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:35.873328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:35.937319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 874/1778] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:25:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:36.067477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e163-bbca-4494-b27c-a93690ef94d1 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:36.079072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:36.080286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:36.080519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:36.080896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] PATH_INFO: `/v3/projects/c77ad00b3a0645daa530ed18d919ea0d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:36.086857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77ad00b3a0645daa530ed18d919ea0d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:36.089247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:36.145434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:36.226144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77ad00b3a0645daa530ed18d919ea0d', 'request_id': 'req-845c8738-daf5-40be-a3fc-187f0a8cbe9b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.242011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77ad00b3a0645daa530ed18d919ea0d', 'request_id': 'req-845c8738-daf5-40be-a3fc-187f0a8cbe9b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.271606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77ad00b3a0645daa530ed18d919ea0d', 'request_id': 'req-845c8738-daf5-40be-a3fc-187f0a8cbe9b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.307679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] The token cache is being invalidate because project c77ad00b3a0645daa530ed18d919ea0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:36.308190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.310338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845c8738-daf5-40be-a3fc-187f0a8cbe9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.317040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 905/1779] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:35 2022] DELETE /identity/v3/projects/c77ad00b3a0645daa530ed18d919ea0d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:36.321000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:36.349872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b71ce-02af-4bab-b072-7c235aef5bde None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:36.546778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:36.548978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:36.549632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:36.550424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] PATH_INFO: `/v3/users/5cabe98f021e42e2855d27ce2dd7f8c2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:36.560394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cabe98f021e42e2855d27ce2dd7f8c2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:36.566647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:36.602674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:36.603118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:36.603465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:36.701217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:36.703562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] MFA Rules not processed for user `ca09fbd6a3b141428edc2473e343b78d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:36.712426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:36.741682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cabe98f021e42e2855d27ce2dd7f8c2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.743146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cabe98f021e42e2855d27ce2dd7f8c2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.746163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cabe98f021e42e2855d27ce2dd7f8c2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.825083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cabe98f021e42e2855d27ce2dd7f8c2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.851695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cabe98f021e42e2855d27ce2dd7f8c2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:36.909723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 875/1780] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:36 2022] DELETE /identity/v3/users/5cabe98f021e42e2855d27ce2dd7f8c2 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:37.008814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 906/1781] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:37.012508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.016384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.173742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.175823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.212797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db425e34-84f9-4f00-a14d-01e601037419 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.249693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.250442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.250731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.251017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.251720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.253839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.355292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 876/1782] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:37 2022] POST /identity/v3/projects => generated 403 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:37.363293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.410260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bfede-2811-467f-96d3-9de70a5adbdd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.522653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:37.611575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1b07-05d6-4c28-903c-ec8e3b0786fc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.627036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.633191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.633521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.634005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.635479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.639406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.679942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 877/1783] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:37 2022] POST /identity/v3/users => generated 308 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:37.684747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.707968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc8f6-a8f5-4733-93c2-ce25c90c6c0c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.714564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.715225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.715675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.715986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.716905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.720533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.720687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.721182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.721467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.721742 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.745658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.756254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 878/1784] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:37.760258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.776948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c43d8-4375-4ec4-81c5-dcba9fc7a019 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.786472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.788136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.788393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.789451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.790003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.790546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] PATH_INFO: `/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a/users/f37b5358605b45309d0af6a12451636a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.819912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 907/1785] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:37.894280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0b245c99f4ead98a91bdcb242dc8a, user_id=f37b5358605b45309d0af6a12451636a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.899071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.910441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:37.934781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 879/1786] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:37 2022] PUT /identity/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a/users/f37b5358605b45309d0af6a12451636a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:37.939331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.960347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a85a6b-7731-41a4-bc4f-ed2f6aa6fbb7 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:38.061781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:38.064319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:38.065055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:38.065616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:38.066668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:38.068491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:38.084499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 908/1787] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:38.086738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:38.104606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae1934-904e-47af-9197-c93d3ca6bdf5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:38.132811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:38.134805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:38.137697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:38.138263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] PATH_INFO: `/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a/users/f37b5358605b45309d0af6a12451636a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:38.147597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0b245c99f4ead98a91bdcb242dc8a, user_id=f37b5358605b45309d0af6a12451636a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:38.160961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13270568-f35d-43e3-bfc0-d6fcd11591f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:38.212494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 880/1788] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:38 2022] PUT /identity/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a/users/f37b5358605b45309d0af6a12451636a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:38.416602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:38.418669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:38.455089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb03b1b-3572-4504-83ec-eebec8bb4082 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:38.635178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:38.636930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:38.641527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:38.642856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] PATH_INFO: `/v3/projects/0075a7ce29994737809711013e52c8b4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:38.644694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0075a7ce29994737809711013e52c8b4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:38.646958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:38.789047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0075a7ce29994737809711013e52c8b4', 'request_id': 'req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:38.807724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:38.809631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0075a7ce29994737809711013e52c8b4', 'request_id': 'req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:38.826021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0075a7ce29994737809711013e52c8b4', 'request_id': 'req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:38.851253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] The token cache is being invalidate because project 0075a7ce29994737809711013e52c8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:38.851889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:38.858204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4cda824d-3fb9-488e-aaca-e6e1ca0fe5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:38.861061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 909/1789] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:38 2022] DELETE /identity/v3/projects/0075a7ce29994737809711013e52c8b4 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:38.865618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:38.866174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:38.866400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:38.920622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:38.928179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:39.127065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 881/1790] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:39.156458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639713b-8980-4971-984e-9f87096c2398 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.159031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639713b-8980-4971-984e-9f87096c2398 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.159618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639713b-8980-4971-984e-9f87096c2398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.195149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1639713b-8980-4971-984e-9f87096c2398 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:39.199315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1639713b-8980-4971-984e-9f87096c2398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:39.214129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 910/1791] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:39.223018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.239312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a955-29d1-4adc-b132-adcd34d5c3f8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.251406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.252549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.253214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.253924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.254965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:39.255364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.257486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.287125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 882/1792] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:39.293680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639713b-8980-4971-984e-9f87096c2398 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.309024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639713b-8980-4971-984e-9f87096c2398 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.316563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.317519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.317886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.318259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.319049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.322227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f34ee3-169c-4bad-86e5-59d65b07d551 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.403046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.403748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 911/1793] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:39.418036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e3b1a-accf-44ef-9902-ab887f6eaf5d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.570383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.570682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.570964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.578259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.582280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.583478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.585731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.589139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.592475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.609465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] MFA Rules not processed for user `7213149649d249f2b2da3d6c7dac0a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:39.617852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:39.654893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 883/1794] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:39.660158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.680473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13238f40-8682-49f3-a2ed-69843a811f99 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.694608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.695597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.695895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.697332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.700080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.705167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.734018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 884/1795] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:39.738401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.767207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f074e-48a4-43c6-97e7-df31b3e86549 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.780739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.781388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.781682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.781947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.783940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.795101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563e53c-8656-4ca6-9258-9c3b0b595bf7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.808809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 912/1796] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:39.808997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 885/1797] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:39.821862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:39.848349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d877-c7cb-441c-b5f6-029fa6d4d8e5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:39.861698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:39.862444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.862755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.863030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] PATH_INFO: `/v3/projects/2b7eb9bcaecf4fc781f67ddfc2db8e30/users/0e617a8e7c7e44828f8886fb99c49369/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.895080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7eb9bcaecf4fc781f67ddfc2db8e30, user_id=0e617a8e7c7e44828f8886fb99c49369, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:39.898346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:39.937279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 913/1798] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:39 2022] PUT /identity/v3/projects/2b7eb9bcaecf4fc781f67ddfc2db8e30/users/0e617a8e7c7e44828f8886fb99c49369/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:39.947362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.947927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.948066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.986867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] MFA Rules not processed for user `0e617a8e7c7e44828f8886fb99c49369`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:39.988345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.025602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:40.091857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.093917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 886/1799] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:40.190942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56862c0-a671-48ce-b20c-7a556a6437f1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.204325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.205580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.206021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.207477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.208379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.213336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60bd60-93e5-49ca-ba29-aac9b33082f7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.276052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 914/1800] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:40 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:40.280587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.295464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5d2e-5b27-43ef-ba8d-62d1ca1c25e9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.346659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.347244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.347529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.379130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] MFA Rules not processed for user `b680411b142642b7acc3585d2481f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:40.381327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.423206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:40.423557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.424290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.424671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.424948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.425716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.427793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.464446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 887/1801] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:40 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:40.465313 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.475026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.493474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f224-caa6-4275-8cff-62ce9ae0a979 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.499045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.499575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 915/1802] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:40.500217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.500491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.500809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.501219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] PATH_INFO: `/v3/users/8a82c36d7946405bad219e9379296861` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.502562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a82c36d7946405bad219e9379296861)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.512874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0f2dc-979c-4f58-882d-74a0809c5181 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.515345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.519388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.520245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.520544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.520821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.521897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.523843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.542942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 916/1803] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:40.550324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.570026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39c0ef-d7e7-4db5-ae44-db7f8514f50f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.586583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.587604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.589920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.591513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] PATH_INFO: `/v3/projects/02ed2470c4474d57b68e24d3404531cf/users/325005fb49ef49c7b157b1b76e07bfd8/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.606418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a82c36d7946405bad219e9379296861'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:40.607115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a82c36d7946405bad219e9379296861'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:40.607809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a82c36d7946405bad219e9379296861'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:40.614071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed2470c4474d57b68e24d3404531cf, user_id=325005fb49ef49c7b157b1b76e07bfd8, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.616879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.625113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a82c36d7946405bad219e9379296861'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:40.636957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b416565c-0cc5-43c8-9679-7ed12fa8c31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a82c36d7946405bad219e9379296861'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:40.639602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 917/1804] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:40 2022] PUT /identity/v3/projects/02ed2470c4474d57b68e24d3404531cf/users/325005fb49ef49c7b157b1b76e07bfd8/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:40.641363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.645744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.661799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 888/1805] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:40 2022] DELETE /identity/v3/users/8a82c36d7946405bad219e9379296861 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:40.666395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.666781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.667178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.687851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:40.689244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.719828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:40.747507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9de2a-e1a8-4966-8981-a670aad8f779 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.769216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 889/1806] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:40.779106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.781348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.817077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.817736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.818018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.818291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.818990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.820674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.832073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 918/1807] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:40.835767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.846841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caebb0-2e84-4091-8092-ba0543a78617 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.852343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.854094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.854499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.854870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] PATH_INFO: `/v3/projects/02ed2470c4474d57b68e24d3404531cf/users/325005fb49ef49c7b157b1b76e07bfd8/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.856992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a005c-9ee8-423a-838d-3d1bea0c9fb2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.859614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed2470c4474d57b68e24d3404531cf, user_id=325005fb49ef49c7b157b1b76e07bfd8, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.862474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37bfaa-ebee-4ba0-a98f-fb8558bf786f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.863276 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.864034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.864410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.865360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.874317 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 919/1808] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:40 2022] PUT /identity/v3/projects/02ed2470c4474d57b68e24d3404531cf/users/325005fb49ef49c7b157b1b76e07bfd8/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:40.885640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.885930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.886211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.887683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.889529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.906657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 890/1809] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:40.922652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] MFA Rules not processed for user `325005fb49ef49c7b157b1b76e07bfd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:40.924232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.949652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:41.020478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 920/1810] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:41.033212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:41.034824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:41.130497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd087ce-1618-426f-9dd2-4c5a691b98d5 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:41.191500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:41.192297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:41.192670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:41.193047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:41.204484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:41.206671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:41.217720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 891/1811] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:41.238500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:41.240834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:41.360487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d421a-b3ff-4cdc-b403-99eedc6dfd67 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:41.365984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:41.366578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:41.366975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:41.367245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] PATH_INFO: `/v3/projects/a65361289774406b8bf06eaba0ae069c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:41.368723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65361289774406b8bf06eaba0ae069c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:41.371070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:41.436984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65361289774406b8bf06eaba0ae069c', 'request_id': 'req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.444159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65361289774406b8bf06eaba0ae069c', 'request_id': 'req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.454921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65361289774406b8bf06eaba0ae069c', 'request_id': 'req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.469728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] The token cache is being invalidate because project a65361289774406b8bf06eaba0ae069c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:41.470058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.470689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-780cbc09-0d71-4576-954a-3d6e7e4dd3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.475405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 921/1812] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:41 2022] DELETE /identity/v3/projects/a65361289774406b8bf06eaba0ae069c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:41.477921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:41.490585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aaaa6-2380-4384-8ea9-09693bd7baa0 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:41.530650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:41.589536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:41.591179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:41.591812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:41.592173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] PATH_INFO: `/v3/users/c776366a686641a08789a3693fd6b92d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:41.594373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c776366a686641a08789a3693fd6b92d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:41.597045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:41.713672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c776366a686641a08789a3693fd6b92d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.714745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c776366a686641a08789a3693fd6b92d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.715475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c776366a686641a08789a3693fd6b92d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.779453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c776366a686641a08789a3693fd6b92d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.793160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c776366a686641a08789a3693fd6b92d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:41.822617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 892/1813] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:41 2022] DELETE /identity/v3/users/c776366a686641a08789a3693fd6b92d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:42.032054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d43d-4d21-4920-bfe8-6a205ab11a64 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:42.032348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d43d-4d21-4920-bfe8-6a205ab11a64 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:42.032626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d43d-4d21-4920-bfe8-6a205ab11a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:42.112340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a579d43d-4d21-4920-bfe8-6a205ab11a64 None None] MFA Rules not processed for user `f37b5358605b45309d0af6a12451636a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:42.113592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a579d43d-4d21-4920-bfe8-6a205ab11a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:42.166140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:42.322941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 922/1814] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:43.253576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:43.258399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:43.411900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f132519-d535-4c72-a24d-97e8f3361935 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:43.417076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:43.417831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.418201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.418584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] PATH_INFO: `/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.420208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b5aae0f9f04a50b3b923d9b32e0638)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:43.422998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:43.497004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b5aae0f9f04a50b3b923d9b32e0638', 'request_id': 'req-f89d4a43-686c-481c-8610-a5d9ca5eac3c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.498377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:43.506142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b5aae0f9f04a50b3b923d9b32e0638', 'request_id': 'req-f89d4a43-686c-481c-8610-a5d9ca5eac3c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.526174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b5aae0f9f04a50b3b923d9b32e0638', 'request_id': 'req-f89d4a43-686c-481c-8610-a5d9ca5eac3c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.557472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] The token cache is being invalidate because project 24b5aae0f9f04a50b3b923d9b32e0638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:43.557769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.558616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.563872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 893/1815] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:43 2022] DELETE /identity/v3/projects/24b5aae0f9f04a50b3b923d9b32e0638 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:43.571854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5af75b-e397-461e-a176-b17e5685e226 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.572416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5af75b-e397-461e-a176-b17e5685e226 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.573525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5af75b-e397-461e-a176-b17e5685e226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.593537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:43.595067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:43.620889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7c5af75b-e397-461e-a176-b17e5685e226 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:43.626153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7c5af75b-e397-461e-a176-b17e5685e226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:43.742190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 923/1816] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:43.743991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.744793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.744793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.782858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4a43-686c-481c-8610-a5d9ca5eac3c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:43.789527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:43.790484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.790945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.791204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:43.791786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] PATH_INFO: `/v3/users/0e617a8e7c7e44828f8886fb99c49369` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.793382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e617a8e7c7e44828f8886fb99c49369)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:43.797987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:43.805490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:43.835217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 924/1817] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:43.836189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:43.849298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca1bc-04cb-4450-b51f-1e9ed77a4603 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:43.863268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:43.866721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.866994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.867264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.868007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:43.870529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:43.874640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:43.897690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e617a8e7c7e44828f8886fb99c49369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.898916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e617a8e7c7e44828f8886fb99c49369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.905838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e617a8e7c7e44828f8886fb99c49369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.906697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 925/1818] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:43.911049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:43.915581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:43.944099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2db6f-6658-4770-8009-0dffe6738e17 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:43.945986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e617a8e7c7e44828f8886fb99c49369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.960637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8af17a48-606c-4645-b992-ea86785433a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e617a8e7c7e44828f8886fb99c49369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:43.960813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:43.961426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.961715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.961992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:43.962673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:43.964625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:43.990465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 894/1819] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:43 2022] DELETE /identity/v3/users/0e617a8e7c7e44828f8886fb99c49369 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:43.997120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207348-dd39-4311-949f-082de0c27f02 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:43.998214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207348-dd39-4311-949f-082de0c27f02 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:43.998660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207348-dd39-4311-949f-082de0c27f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.036429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 926/1820] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:43 2022] POST /identity/v3/projects => generated 399 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:44.040739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.056781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6b207348-dd39-4311-949f-082de0c27f02 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:44.060395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6b207348-dd39-4311-949f-082de0c27f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:44.216652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd351f-d4de-4c95-a88a-bda8db9c6f3c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.229825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.230751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.231170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.231576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.232630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.237851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.251288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:44.265101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 895/1821] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:44.282986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83778-6783-40c6-962d-dae003a68d72 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.283309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83778-6783-40c6-962d-dae003a68d72 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.283602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83778-6783-40c6-962d-dae003a68d72 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.286606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 896/1822] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:25:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:44.303391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 927/1823] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:44 2022] POST /identity/v3/users => generated 300 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:44.307797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a83778-6783-40c6-962d-dae003a68d72 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.337978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a83778-6783-40c6-962d-dae003a68d72 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.349943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.350576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.350853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.351125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.351824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.354322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.378709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 897/1824] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:44.383266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.384819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.386579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:44.396911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41758-85e9-4e97-8e8b-2290e26f29d3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.403835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.405268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.405268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.405461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.407976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.408368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf08517-6028-4510-ac25-9b6e581033d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.425442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 898/1825] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:44.425806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:44.427999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.428370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.428654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.459777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:44.461356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:44.502328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725276-9d3d-4066-877f-58360ed4210f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.510311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.511420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.511593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.513318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.542848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.545230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.554004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 899/1826] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:25:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:44.555365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.559551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 928/1827] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:44.569655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b5a16-24d7-4bba-bae4-cc084ed347aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.574941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.575621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.575931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.576246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] PATH_INFO: `/v3/projects/c667e8c4917e4311ace33a84cf1d098e/users/cd6ae1f8718d493fa83b7409a0a1028c/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.587744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.590658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:44.597757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c667e8c4917e4311ace33a84cf1d098e, user_id=cd6ae1f8718d493fa83b7409a0a1028c, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.615597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc045825-872e-400b-aeb5-a1ae013d2104 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.644465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6deadcd-b743-4004-9a7b-0cad9ee2ddff service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:44.658107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:44.660273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.664024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.664357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.693415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 900/1828] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:44 2022] PUT /identity/v3/projects/c667e8c4917e4311ace33a84cf1d098e/users/cd6ae1f8718d493fa83b7409a0a1028c/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:44.705245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:44.705762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:44.706070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:44.706560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:44.707764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:44.728213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 929/1829] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:44.779248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] MFA Rules not processed for user `cd6ae1f8718d493fa83b7409a0a1028c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:44.780884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:44.938958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 901/1830] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:44.943011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:44.954222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe022d-1472-4b5d-b7ac-02074fd9fa56 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.100449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.101067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.101353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.101632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.102308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.104727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.184408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 930/1831] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:44 2022] POST /identity/v3/projects => generated 401 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:45.189329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.205308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f7d72-6e09-4a68-90b9-f0a7de958964 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.289698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:45.321553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.322757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.322757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.323014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.323784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.325984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.354625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 902/1832] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:45 2022] POST /identity/v3/users => generated 307 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:45.359538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.382088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e664ba-266d-4962-a027-56eaf55a4ad4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.387104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.387844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.388376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.388734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.390638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.391897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.403546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 931/1833] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:45.409198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.409669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.411584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:45.419641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c134cb5-950f-4de6-a34e-476563e83fd8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.425016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20308f4-f627-4023-bee1-3f888f92d151 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.425568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.426265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.426548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.426852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] PATH_INFO: `/v3/projects/06dc52a46615467ea80c59d8286a41d1/users/0e73efd331a6418eb456e55013a26d50/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.430395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.431189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.431580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.431944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] PATH_INFO: `/v3/projects/2b7eb9bcaecf4fc781f67ddfc2db8e30` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.433231 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7eb9bcaecf4fc781f67ddfc2db8e30)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.435298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.443521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06dc52a46615467ea80c59d8286a41d1, user_id=0e73efd331a6418eb456e55013a26d50, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.446439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.465373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 903/1834] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:45 2022] PUT /identity/v3/projects/06dc52a46615467ea80c59d8286a41d1/users/0e73efd331a6418eb456e55013a26d50/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:45.470018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.510214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7eb9bcaecf4fc781f67ddfc2db8e30', 'request_id': 'req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.517311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7eb9bcaecf4fc781f67ddfc2db8e30', 'request_id': 'req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.527310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7eb9bcaecf4fc781f67ddfc2db8e30', 'request_id': 'req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.536034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:45.548651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] The token cache is being invalidate because project 2b7eb9bcaecf4fc781f67ddfc2db8e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:45.548947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.549972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.555736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 932/1835] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:45 2022] DELETE /identity/v3/projects/2b7eb9bcaecf4fc781f67ddfc2db8e30 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:45.555902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd1a3-cb48-464f-b575-50d513838e04 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.556890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.558348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:45.629962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30d3-bbf5-4135-9794-91ce3d8139dd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.633424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.634383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.634601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.635297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.636457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.637324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.637739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.638160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] PATH_INFO: `/v3/users/325005fb49ef49c7b157b1b76e07bfd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.640056 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=325005fb49ef49c7b157b1b76e07bfd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.643263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.645469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.684140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 904/1836] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:45.684140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.687684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:45.707447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e365-6526-489c-b012-86f545e969d0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:45.731031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:45.732000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:45.732274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:45.732557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] PATH_INFO: `/v3/users/ca09fbd6a3b141428edc2473e343b78d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:45.737678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca09fbd6a3b141428edc2473e343b78d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:45.739632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:45.764379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325005fb49ef49c7b157b1b76e07bfd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.769809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325005fb49ef49c7b157b1b76e07bfd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.774192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325005fb49ef49c7b157b1b76e07bfd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.800789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325005fb49ef49c7b157b1b76e07bfd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.815797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325005fb49ef49c7b157b1b76e07bfd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.839337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:45.846617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 933/1837] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:45 2022] DELETE /identity/v3/users/325005fb49ef49c7b157b1b76e07bfd8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:45.847754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:45.849339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:45.850436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca09fbd6a3b141428edc2473e343b78d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.851671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca09fbd6a3b141428edc2473e343b78d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.853863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca09fbd6a3b141428edc2473e343b78d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.877963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca09fbd6a3b141428edc2473e343b78d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.899321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9da57e82-e769-4139-a675-59abbe6e84f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca09fbd6a3b141428edc2473e343b78d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:45.927498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 905/1838] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:45 2022] DELETE /identity/v3/users/ca09fbd6a3b141428edc2473e343b78d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:45.980653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567917b1-44ea-4aaa-962f-4805b7860ecd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:46.009645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:46.065922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:46.067520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.068043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.068614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] PATH_INFO: `/v3/projects/06dc52a46615467ea80c59d8286a41d1/users/0e73efd331a6418eb456e55013a26d50/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.071940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=06dc52a46615467ea80c59d8286a41d1, user_id=0e73efd331a6418eb456e55013a26d50, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:46.075459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:46.090816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 934/1839] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:45 2022] PUT /identity/v3/projects/06dc52a46615467ea80c59d8286a41d1/users/0e73efd331a6418eb456e55013a26d50/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:46.101442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.101989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.102439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.147319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] MFA Rules not processed for user `0e73efd331a6418eb456e55013a26d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:46.149072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:46.224067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 906/1840] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:46.235538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:46.237856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:46.316769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3e13d-e8a4-45a0-a5cd-130014fe49bf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:46.321770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:46.322752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.323134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.323640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.336709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:46.338470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8af9cd-66d4-40a0-8005-fab11cd77c44 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:46.361305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 935/1841] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:46.476858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:46.503315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0d83e-d6dd-41c1-925b-c9b8a8c16092 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:46.517474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:46.517474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.517474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.517474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.523035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:46.657385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:46.659739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d08f99-c44d-4d20-bbf3-f2983e3e4c02 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:46.687273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 907/1842] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:46.787885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.788409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.788511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.790660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 936/1843] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:25:46.816273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:46.818115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:46.818495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:46.861407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:46.864270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:46.976259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:47.065220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 908/1844] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:47.371012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:47.373132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:47.412883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:47.414374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:47.568466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2793b-7217-4611-8a67-5b60eae4d7fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:47.588559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8605e4-427f-40a1-95ac-a1a4421c0a9e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:47.589236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:47.589922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:47.590289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:47.590656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] PATH_INFO: `/v3/projects/02ed2470c4474d57b68e24d3404531cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:47.595665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ed2470c4474d57b68e24d3404531cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:47.603817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:47.609297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:47.610069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:47.610359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:47.610631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] PATH_INFO: `/v3/projects/71cd0200d71c477f8953d99a3500a588` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:47.611980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71cd0200d71c477f8953d99a3500a588)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:47.614038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:47.708778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ed2470c4474d57b68e24d3404531cf', 'request_id': 'req-02d54fe4-37aa-4a3b-9b80-6365070e9597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.716357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ed2470c4474d57b68e24d3404531cf', 'request_id': 'req-02d54fe4-37aa-4a3b-9b80-6365070e9597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.721992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:47.726415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71cd0200d71c477f8953d99a3500a588', 'request_id': 'req-9163677f-b525-4f16-8b39-38ef795ddf3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.732188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ed2470c4474d57b68e24d3404531cf', 'request_id': 'req-02d54fe4-37aa-4a3b-9b80-6365070e9597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.749201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] The token cache is being invalidate because project 02ed2470c4474d57b68e24d3404531cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:47.749652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.750568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.752849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 937/1845] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:47 2022] DELETE /identity/v3/projects/02ed2470c4474d57b68e24d3404531cf => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:47.753691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:47.756358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:47.757521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71cd0200d71c477f8953d99a3500a588', 'request_id': 'req-9163677f-b525-4f16-8b39-38ef795ddf3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.786106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71cd0200d71c477f8953d99a3500a588', 'request_id': 'req-9163677f-b525-4f16-8b39-38ef795ddf3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.825790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] The token cache is being invalidate because project 71cd0200d71c477f8953d99a3500a588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:47.827701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.830173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9163677f-b525-4f16-8b39-38ef795ddf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:47.835495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 909/1846] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:47 2022] DELETE /identity/v3/projects/71cd0200d71c477f8953d99a3500a588 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:47.844299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ddac8-facd-44b7-851e-41c693789568 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:47.845746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ddac8-facd-44b7-851e-41c693789568 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:47.846367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ddac8-facd-44b7-851e-41c693789568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:47.885401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc4ddac8-facd-44b7-851e-41c693789568 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:47.887645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc4ddac8-facd-44b7-851e-41c693789568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:47.927538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d54fe4-37aa-4a3b-9b80-6365070e9597 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:47.937456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:47.938509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:47.938953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:47.939378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:47.941659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:47.966476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:47.969195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:47.996802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 938/1847] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:47.998202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.005486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:48.061160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 910/1848] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:48.069558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.069854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.070322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.090969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:48.114252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:48.133746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:48.153548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 911/1849] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:48.157898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.177143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3525b-e17b-4456-a9df-4780de5b6fdd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.190361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.191045 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.204698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.205720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054229c7-6556-40a5-a6a6-046acf265993 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.206605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.207396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:48.207720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.222851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.229526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.229738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.230294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.230770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] PATH_INFO: `/v3/users/f37b5358605b45309d0af6a12451636a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.232616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37b5358605b45309d0af6a12451636a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.237574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.280569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 912/1850] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:25:48.284942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.308379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ab0cb-8a43-4c3d-ae02-78eebfac827a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.317957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.318719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.319004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.321432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.322203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.326546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.354732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37b5358605b45309d0af6a12451636a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:48.355741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37b5358605b45309d0af6a12451636a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:48.356724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37b5358605b45309d0af6a12451636a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:48.372700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37b5358605b45309d0af6a12451636a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:48.381544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37b5358605b45309d0af6a12451636a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:48.385731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:48.399374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 939/1851] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:47 2022] DELETE /identity/v3/users/f37b5358605b45309d0af6a12451636a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:48.453764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 913/1852] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:48 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:48.457731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.460241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:48.601221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e722b-8cda-424a-a262-37b17d14b7c5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.701083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.702359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.702655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.702979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.703883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.706062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.748159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 940/1853] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:48 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:48.749907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.766018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c70dc-a541-4a3f-8164-f52db81e3a66 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.773456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.775536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.776043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.776660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.778799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.781482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.798134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 914/1854] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:48.802864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.820164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa75461-2819-4792-b9f4-8b421900a50f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.833230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.833990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.834378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.835010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.836337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.838514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c6c4d-b540-4ae6-8ebf-b835f8d45b1d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.872598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 941/1855] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:48.877863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:48.897459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293527-1d70-4097-bbdd-a082ecc906ed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:48.903206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:48.904321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.904705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.905707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] PATH_INFO: `/v3/projects/9def2c281e3a4ebe8509a4f684abb8a9/users/071cb5d9eb1a40b2be6098c6785f13e2/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:48.925474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9def2c281e3a4ebe8509a4f684abb8a9, user_id=071cb5d9eb1a40b2be6098c6785f13e2, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:48.939944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:48.975855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:48.976318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 915/1856] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:48 2022] PUT /identity/v3/projects/9def2c281e3a4ebe8509a4f684abb8a9/users/071cb5d9eb1a40b2be6098c6785f13e2/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:48.976413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:48.976661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.033038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] MFA Rules not processed for user `071cb5d9eb1a40b2be6098c6785f13e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:49.033038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:49.159476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 942/1857] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:49.162213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.185866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.191629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:49.217103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:49.316857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d572ca5-5598-46fc-ad7a-cebc32534829 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.325239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.326202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.326727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.327749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] PATH_INFO: `/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.330148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b0b245c99f4ead98a91bdcb242dc8a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.333006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.409982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270558b-339f-4c55-822b-8dae0cc941fb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.421132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.421900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.422179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.422715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.424636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.426556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.443531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b0b245c99f4ead98a91bdcb242dc8a', 'request_id': 'req-3ca7c440-682c-44b3-ba45-9505df346569'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:49.449826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b0b245c99f4ead98a91bdcb242dc8a', 'request_id': 'req-3ca7c440-682c-44b3-ba45-9505df346569'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:49.459183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b0b245c99f4ead98a91bdcb242dc8a', 'request_id': 'req-3ca7c440-682c-44b3-ba45-9505df346569'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:49.470302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 916/1858] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/projects => generated 387 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:49.471486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.475866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] The token cache is being invalidate because project a5b0b245c99f4ead98a91bdcb242dc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:49.476486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:49.477542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3ca7c440-682c-44b3-ba45-9505df346569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:49.479609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 943/1859] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:49 2022] DELETE /identity/v3/projects/a5b0b245c99f4ead98a91bdcb242dc8a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:49.484360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2383e4-8376-4cb5-b666-c40dc5c802d9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.485221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2383e4-8376-4cb5-b666-c40dc5c802d9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.485707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2383e4-8376-4cb5-b666-c40dc5c802d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.521784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dd2383e4-8376-4cb5-b666-c40dc5c802d9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:49.523139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dd2383e4-8376-4cb5-b666-c40dc5c802d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:49.565955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5513d-a103-4db9-bba1-8c3af93ce6c8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.572470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.573315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.573315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.573605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.574402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.578175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.603998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 944/1860] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:49.614072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.615705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.616434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.623160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 917/1861] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:49.629114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.636265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:49.656683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:49.663221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d2dd9-9d61-4a2f-b1b7-f4d125b085a3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.665013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:49.667305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:49.669633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.670721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.671130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.671719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.673183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.675354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.678852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 945/1862] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:49.683800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.695271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 918/1863] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:49.696828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.702533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b78a8b-5552-46d3-b96b-96308cbf3b4c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.708671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.708876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.709158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.709344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6190-ec93-4fce-9114-c4668e25146d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.709994 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.710824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:49.711234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.713174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.715178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.716113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.716517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.716891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] PATH_INFO: `/v3/projects/182cab572b3f45a1986b0c968098bc95/users/cd67e68e31b54f53a4339b049c517dd0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.728426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 946/1864] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:49.732818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.739457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182cab572b3f45a1986b0c968098bc95, user_id=cd67e68e31b54f53a4339b049c517dd0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.743079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.745152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30f9a1-6f10-4f50-a8eb-c428c4268cf2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.751678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.752356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.752790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.753073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.753923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.756453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.765801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 919/1865] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:49 2022] PUT /identity/v3/projects/182cab572b3f45a1986b0c968098bc95/users/cd67e68e31b54f53a4339b049c517dd0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:49.768077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.786445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:49.805305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 947/1866] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/projects => generated 363 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:49.809505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.858587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed0be-8892-4afd-b7a2-9108dbfa93c8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.926269 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf068c97-2493-4bb7-8e63-954fb7e63ce5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.930805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.931430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.931870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.932178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.933470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.935165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.939062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.939695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.940005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.940298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.941125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.942739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.954570 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 920/1867] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:49.958756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.960856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 948/1868] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:49 2022] POST /identity/v3/users => generated 282 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:49.964985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:49.969456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ab9ca-1026-4d8c-9c43-ce49470e665f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.977462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34267a9a-f697-4686-a3e8-ed6ef3032e30 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:49.977975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.978478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.978775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.979033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] PATH_INFO: `/v3/projects/182cab572b3f45a1986b0c968098bc95/users/cd67e68e31b54f53a4339b049c517dd0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.981019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182cab572b3f45a1986b0c968098bc95, user_id=cd67e68e31b54f53a4339b049c517dd0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.982858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:49.984143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:49.984263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.984603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:49.984974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:49.985768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:49.988242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed17e2-5bf3-4f99-92c4-c4d455570d94 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:49.995618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 921/1869] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:49 2022] PUT /identity/v3/projects/182cab572b3f45a1986b0c968098bc95/users/cd67e68e31b54f53a4339b049c517dd0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:50.002115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 949/1870] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:50.008341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.012617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.012617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.013281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.060381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] MFA Rules not processed for user `cd67e68e31b54f53a4339b049c517dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:50.063653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:50.126054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:50.137155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be710ff-d11d-4a88-a155-14c9a188c0e0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.148635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:50.149271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.149548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.149816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.150717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:50.153414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:50.197257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 922/1871] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:50.198569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.219808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77687f86-4d09-4b19-812c-0ba1b271e37b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.225839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:50.227177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.227808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.228448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] PATH_INFO: `/v3/projects/97f388eafa6544a7b966af69910652c4/users/fdaf221f93ba492db9a80ea58d5f82c9/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.235038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 950/1872] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:50.251077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.255909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:50.288627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f388eafa6544a7b966af69910652c4, user_id=fdaf221f93ba492db9a80ea58d5f82c9, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:50.300059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1d384-d603-42a6-bb80-acada482b9d2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:50.324766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 923/1873] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:50 2022] PUT /identity/v3/projects/97f388eafa6544a7b966af69910652c4/users/fdaf221f93ba492db9a80ea58d5f82c9/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:50.341703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.345882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.348429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.439822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] MFA Rules not processed for user `fdaf221f93ba492db9a80ea58d5f82c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:50.441725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:50.478637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f29e-ef7a-45ad-848a-7bd43cece45e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.528788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:50.576199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 924/1874] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:50.579902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.585057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:50.585960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.586432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.586910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.696233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:50.698750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:50.714417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 951/1875] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:50.740314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e532d-30d2-4bba-a30f-eb025dd922eb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.758725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.760132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:50.760791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.761082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.761349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.762049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:50.764091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:50.775850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10105f12-538c-4a1f-a4f8-7f171977e7af service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.784514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:50.785461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.785931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.786363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] PATH_INFO: `/v3/users/071cb5d9eb1a40b2be6098c6785f13e2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:50.787980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] RBAC: Authorizing `identity:delete_user(user_id=071cb5d9eb1a40b2be6098c6785f13e2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:50.791416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:50.826005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 925/1876] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:50 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:50.831299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:50.901726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071cb5d9eb1a40b2be6098c6785f13e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:50.902921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071cb5d9eb1a40b2be6098c6785f13e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:50.903802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071cb5d9eb1a40b2be6098c6785f13e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:50.936133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071cb5d9eb1a40b2be6098c6785f13e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:50.944156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:50.960726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-04c3e030-6df0-4835-ab7a-ab9f671d802c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071cb5d9eb1a40b2be6098c6785f13e2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:50.985217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9c446-6ffe-44b0-95a3-c949ce9b3f33 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:50.985938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 952/1877] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:50 2022] DELETE /identity/v3/users/071cb5d9eb1a40b2be6098c6785f13e2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:50.987722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:50.987980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:50.988164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.020308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:51.021829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.074648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.075292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.075574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.075848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.076586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.079858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.085839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:51.107977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 926/1878] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:50 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:51.111839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.113590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.124408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 953/1879] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:51.129870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e482a2a-c91e-4497-8628-2da856603e71 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.134841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.135533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.135878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.136266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.137084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.138829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.152576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 927/1880] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:51.157355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.168244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219ba5-dab8-4f6e-9fac-bf6abac13d42 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.174004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.174847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.175414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.175724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] PATH_INFO: `/v3/projects/dd4ad84b0048403697cf20d6b1646235/users/a9ccd215fcdd4879b634efa1db348bc2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.192621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4ad84b0048403697cf20d6b1646235, user_id=a9ccd215fcdd4879b634efa1db348bc2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.195406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.210437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 954/1881] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:51 2022] PUT /identity/v3/projects/dd4ad84b0048403697cf20d6b1646235/users/a9ccd215fcdd4879b634efa1db348bc2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:51.213964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.224586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.226434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.230760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231a04f-d329-4482-adfc-ee4cf5f540dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.325417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df536b04-e71c-4bee-87ae-00a8aa48a8a5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.328899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.329979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.330939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.331234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.381408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.382267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.382647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.383144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.385691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.388489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.406205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.407744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7a388-b749-42e2-bf81-ad49b1d48dca service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.409472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 928/1882] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:51.413319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.421351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 955/1883] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:51.429966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26952-94cc-4257-89c6-046c2b72042a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.435305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.436180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.437687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.438040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] PATH_INFO: `/v3/projects/dd4ad84b0048403697cf20d6b1646235/users/a9ccd215fcdd4879b634efa1db348bc2/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.441194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4ad84b0048403697cf20d6b1646235, user_id=a9ccd215fcdd4879b634efa1db348bc2, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.443243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.469743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 929/1884] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:51 2022] PUT /identity/v3/projects/dd4ad84b0048403697cf20d6b1646235/users/a9ccd215fcdd4879b634efa1db348bc2/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:51.479669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.479866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.480168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.518287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] MFA Rules not processed for user `a9ccd215fcdd4879b634efa1db348bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:51.519642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.596135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:51.611521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 956/1885] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:51.622631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.624473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.719329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd9d2-2028-4fe4-9d3d-0f07b58984ac admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.729216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.731483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.731483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.731483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.735915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.737300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.751397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:51.754578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.757231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.791979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 930/1886] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:51.834372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999e329-a1b1-4304-ab8f-f7fd1a7ffccc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.842497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.843271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.843735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.844115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] PATH_INFO: `/v3/users/cd6ae1f8718d493fa83b7409a0a1028c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.845634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6ae1f8718d493fa83b7409a0a1028c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:51.847388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:51.894978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:51.919461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7a94-46e9-44e3-92ea-d04db36466ba service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:51.933878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6ae1f8718d493fa83b7409a0a1028c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:51.934136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:51.934916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6ae1f8718d493fa83b7409a0a1028c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:51.935560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.935798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6ae1f8718d493fa83b7409a0a1028c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:51.936082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.937079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:51.940800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:51.950777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6ae1f8718d493fa83b7409a0a1028c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:51.960559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-865afe6c-9626-401f-b824-94f32f7936e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6ae1f8718d493fa83b7409a0a1028c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:51.976791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 957/1887] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:51 2022] DELETE /identity/v3/users/cd6ae1f8718d493fa83b7409a0a1028c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:51.981594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:51.982113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:51.982504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:52.021857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:52.026416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:52.079914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:52.081532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:52.092122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 931/1888] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:52.092848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:52.094225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:52.108783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f415e6-9e3a-4434-a2fd-b81b57f55fc0 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:52.133700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 958/1889] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:52.144211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:52.145819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:52.161181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:52.182677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:52.183799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:52.184344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:52.184861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] PATH_INFO: `/v3/projects/9def2c281e3a4ebe8509a4f684abb8a9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:52.187146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9def2c281e3a4ebe8509a4f684abb8a9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:52.189108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:52.217043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904dd6b-8575-4375-a8a8-16eb22f4bf81 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:52.221549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:52.222274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:52.222799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:52.223173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:52.245237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:52.247333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:52.259268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9def2c281e3a4ebe8509a4f684abb8a9', 'request_id': 'req-e4e8d69a-4092-4a6a-a504-923c5476a6d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.262249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 959/1890] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:52.278186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9def2c281e3a4ebe8509a4f684abb8a9', 'request_id': 'req-e4e8d69a-4092-4a6a-a504-923c5476a6d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.284795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9def2c281e3a4ebe8509a4f684abb8a9', 'request_id': 'req-e4e8d69a-4092-4a6a-a504-923c5476a6d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.316347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] The token cache is being invalidate because project 9def2c281e3a4ebe8509a4f684abb8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:52.316787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.317769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.319657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 932/1891] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:52 2022] DELETE /identity/v3/projects/9def2c281e3a4ebe8509a4f684abb8a9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:52.324024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:52.337568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ac4-85ff-46a4-b85a-de88cd2782b7 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:52.408844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:52.409417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:52.409697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:52.409978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] PATH_INFO: `/v3/users/cd67e68e31b54f53a4339b049c517dd0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:52.411584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd67e68e31b54f53a4339b049c517dd0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:52.413860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:52.491011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd67e68e31b54f53a4339b049c517dd0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.493271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd67e68e31b54f53a4339b049c517dd0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.494077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd67e68e31b54f53a4339b049c517dd0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.515480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd67e68e31b54f53a4339b049c517dd0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.526593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd67e68e31b54f53a4339b049c517dd0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.553781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 960/1892] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:52 2022] DELETE /identity/v3/users/cd67e68e31b54f53a4339b049c517dd0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:52.638428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:52.640901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:52.661190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8d69a-4092-4a6a-a504-923c5476a6d6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:52.722103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:52.802972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:52.804868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:52.805589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:52.806078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] PATH_INFO: `/v3/projects/c667e8c4917e4311ace33a84cf1d098e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:52.815661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c667e8c4917e4311ace33a84cf1d098e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:52.820310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:52.926234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c667e8c4917e4311ace33a84cf1d098e', 'request_id': 'req-0a5d0137-1681-4f7b-ae9f-605ea05420fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.932701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c667e8c4917e4311ace33a84cf1d098e', 'request_id': 'req-0a5d0137-1681-4f7b-ae9f-605ea05420fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.943730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c667e8c4917e4311ace33a84cf1d098e', 'request_id': 'req-0a5d0137-1681-4f7b-ae9f-605ea05420fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.960041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] The token cache is being invalidate because project c667e8c4917e4311ace33a84cf1d098e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:52.960514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.961254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:52.967409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 933/1893] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:52 2022] DELETE /identity/v3/projects/c667e8c4917e4311ace33a84cf1d098e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:52.968472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:52.969987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:52.984586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724050d-b61f-430c-98dc-e2c24196037a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.019888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:53.051025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.051695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.051810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.052110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] PATH_INFO: `/v3/users/0e73efd331a6418eb456e55013a26d50` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.053653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e73efd331a6418eb456e55013a26d50)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.055924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:53.116957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e73efd331a6418eb456e55013a26d50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.117623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e73efd331a6418eb456e55013a26d50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.118765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e73efd331a6418eb456e55013a26d50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.135673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e73efd331a6418eb456e55013a26d50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.144111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e73efd331a6418eb456e55013a26d50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.164916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 961/1894] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:52 2022] DELETE /identity/v3/users/0e73efd331a6418eb456e55013a26d50 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:53.319587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:53.321873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.338252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0137-1681-4f7b-ae9f-605ea05420fb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.430439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.431205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.431582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.431979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] PATH_INFO: `/v3/projects/182cab572b3f45a1986b0c968098bc95` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.433640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] RBAC: Authorizing `identity:delete_project(project_id=182cab572b3f45a1986b0c968098bc95)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.435895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:53.496997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182cab572b3f45a1986b0c968098bc95', 'request_id': 'req-86941c33-aab5-4001-9735-7813156f311e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.504465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:53.508455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182cab572b3f45a1986b0c968098bc95', 'request_id': 'req-86941c33-aab5-4001-9735-7813156f311e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.524214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182cab572b3f45a1986b0c968098bc95', 'request_id': 'req-86941c33-aab5-4001-9735-7813156f311e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.543045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:53.543218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] The token cache is being invalidate because project 182cab572b3f45a1986b0c968098bc95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:53.543462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.544615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-86941c33-aab5-4001-9735-7813156f311e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.545667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.546626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 934/1895] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:53 2022] DELETE /identity/v3/projects/182cab572b3f45a1986b0c968098bc95 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:53.551279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1e4e-c8fb-4fa5-a301-fec71699b420 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.551670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1e4e-c8fb-4fa5-a301-fec71699b420 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.551972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1e4e-c8fb-4fa5-a301-fec71699b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.568885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2828a7-ffb6-4a93-9467-62bd1ecf63fc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.592579 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-20fe1e4e-c8fb-4fa5-a301-fec71699b420 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:53.593809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-20fe1e4e-c8fb-4fa5-a301-fec71699b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.640367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.640974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.641259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.641583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] PATH_INFO: `/v3/projects/06dc52a46615467ea80c59d8286a41d1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.642935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06dc52a46615467ea80c59d8286a41d1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.645039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:53.667362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 935/1896] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:53.694086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.698753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.699081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.729922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06dc52a46615467ea80c59d8286a41d1', 'request_id': 'req-989b7f99-7531-4eed-a73b-91ba8b8a71d8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.741393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:53.743199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.744666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:53.747626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06dc52a46615467ea80c59d8286a41d1', 'request_id': 'req-989b7f99-7531-4eed-a73b-91ba8b8a71d8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.762391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06dc52a46615467ea80c59d8286a41d1', 'request_id': 'req-989b7f99-7531-4eed-a73b-91ba8b8a71d8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.779203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] The token cache is being invalidate because project 06dc52a46615467ea80c59d8286a41d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:53.779368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.780110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-989b7f99-7531-4eed-a73b-91ba8b8a71d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:53.785912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 962/1897] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:53 2022] DELETE /identity/v3/projects/06dc52a46615467ea80c59d8286a41d1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:53.787441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007f4b0-a361-4d2a-b18c-a39ec269f388 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.787738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007f4b0-a361-4d2a-b18c-a39ec269f388 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.788312 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007f4b0-a361-4d2a-b18c-a39ec269f388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.812067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9007f4b0-a361-4d2a-b18c-a39ec269f388 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:53.813498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9007f4b0-a361-4d2a-b18c-a39ec269f388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.886995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 963/1898] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:53.897198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.897538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.897816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.901615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 936/1899] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:53.906979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:53.916432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6667a-59ee-4f03-81c1-75fa2b24fc6c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.922594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.923155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.923434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.923706 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.924542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:53.924843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.926634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:53.931654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:53.932991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:53.941776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 937/1900] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:53.944236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 964/1901] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:53.948723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:53.948895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:53.960452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532493ea-19c7-421a-affb-e7f8dd56130b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.960634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25334d4-87a5-4d51-b5c1-c569d6849889 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:53.966323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.968542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.968907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.969214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.969925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:53.970227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.972403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:53.972644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:53.973028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:53.973370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:53.973697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:53.974240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:53.976163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.001457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 965/1902] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:54.005534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.012642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 938/1903] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:53 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:54.014879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.017266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2566535-778c-4f3b-ba25-96447ab2a21e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.081783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.082473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.082663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.082936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.083593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.084317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:54.085526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.106226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455d372-880d-407e-b13c-9d25ed37dac8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.120525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.121434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.121845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.122260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.123164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.125656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 966/1904] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:54.126095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.129658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.165804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 939/1905] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:54.170953 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.188627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a93-579f-4bb0-8246-2aa9b5b7d11c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.197648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:54.211939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafabc1-2ae0-4b0d-863e-dbcd4499e636 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.222176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.223140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.223756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.223987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.225014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.229202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.271992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 967/1906] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:25:54.273275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.278451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.279442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.279862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.280806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.281728 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.283680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.287198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0db540-9610-44ba-9aed-34342b5d169b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.293120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.293829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.294168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.294524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.295488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.298163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.301668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 940/1907] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:54.302842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.315433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d53a-96a5-42ef-b4b2-e1a47d69dbfe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.322627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.323429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.324185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.324473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.325177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.328106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.332373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 968/1908] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:54.347113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 941/1909] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:54.349065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.350085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.367102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83320cc7-2a21-429b-ab71-3bf973cd3036 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.370567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56626-534a-43d4-be96-24d43b491cd6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.378478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.379210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.379497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.379776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] PATH_INFO: `/v3/projects/5a3f698a181841799da576aa3e4b630b/users/ebbac80e8ee34202bc28ee0f65e67ee9/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.382404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.384597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.385253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.385748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.386697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.388539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.400737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3f698a181841799da576aa3e4b630b, user_id=ebbac80e8ee34202bc28ee0f65e67ee9, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.403635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a4978-aedd-47d9-8dfe-af3ea12bfd96 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.419228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 969/1910] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:54.421539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.433201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8793-7ec2-4622-9dae-32700c469e57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.439249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 942/1911] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:54 2022] PUT /identity/v3/projects/5a3f698a181841799da576aa3e4b630b/users/ebbac80e8ee34202bc28ee0f65e67ee9/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:54.440755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.441397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.441995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.477561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] MFA Rules not processed for user `ebbac80e8ee34202bc28ee0f65e67ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:54.480297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:54.546325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.547771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.548306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.548787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] PATH_INFO: `/v3/projects/f11bb5290b2c4df6948fedffeb903104/users/68bab262c80b49a882cabefcfe3ccd9e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.566495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11bb5290b2c4df6948fedffeb903104, user_id=68bab262c80b49a882cabefcfe3ccd9e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.570082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544b0d-5873-4be0-90f8-87a51415b03e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.584392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 943/1912] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:54.588895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.592164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 970/1913] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:54 2022] PUT /identity/v3/projects/f11bb5290b2c4df6948fedffeb903104/users/68bab262c80b49a882cabefcfe3ccd9e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:54.602527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c191-947f-4b09-a2cf-bf1fac30e9ba None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.603878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.604267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.604630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.649467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:54.668907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] MFA Rules not processed for user `68bab262c80b49a882cabefcfe3ccd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:54.670143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:54.715750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.718794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.719106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.719424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.720404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.724819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.765025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 971/1914] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:54.768210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 944/1915] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:54.769077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.769211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.783258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79a80-6804-4d46-a857-12d975e8be1b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.836869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:54.886166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754e36-1a15-459b-8886-6adfbcff243a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:54.889089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.890458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.891306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.891584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.892294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.892824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:54.893555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:54.893898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:54.894242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:54.894814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.895127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:54.897115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:54.933652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 945/1916] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:54.938252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 972/1917] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:54.939753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.939869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:54.957554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ed8e2-ce65-4165-b760-8f177c18ba66 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.028127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.028641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.028932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.029746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.030455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.032907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.045641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 973/1918] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:55.050403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.050697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4fe32-2afc-4926-867e-288230b2a058 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.056882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.057518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.057800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.058118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.058751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.060813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.083870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb316-a87a-47a6-92f0-8d20081f7406 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.105176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.105176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.105458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.105580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] PATH_INFO: `/v3/projects/593c4a93cb9a442a9d2f712d0942e950/users/9080d764164c443781c28be667a15b31/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.139524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 946/1919] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:54 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:55.139782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=593c4a93cb9a442a9d2f712d0942e950, user_id=9080d764164c443781c28be667a15b31, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.143469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.143730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.171328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 974/1920] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:55 2022] PUT /identity/v3/projects/593c4a93cb9a442a9d2f712d0942e950/users/9080d764164c443781c28be667a15b31/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:55.173140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.175429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20149-c804-426b-89f5-15de91eeeafc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.250671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:55.308367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.309670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.310224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.310693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.313667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.316143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.330921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 947/1921] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:55.335396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.335925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1e41-f1da-4b76-bb10-07bcf45ba7cf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.348076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.349384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.349876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.351845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.352926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.356476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.358124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5731a-35c8-4808-bb53-775d21af9be5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.368029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.370295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.370507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.370770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] PATH_INFO: `/v3/projects/c54db5bc40e141fe91de819c80a28473/users/7cdbc4270cac4de69f73c654c6eb085d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.380170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 975/1922] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:55.385359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.401437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a749d-6648-4c48-ba78-cc3ca34188b2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.407278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54db5bc40e141fe91de819c80a28473, user_id=7cdbc4270cac4de69f73c654c6eb085d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.414268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.418192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.420391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.421513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.421815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] PATH_INFO: `/v3/projects/593c4a93cb9a442a9d2f712d0942e950/users/9080d764164c443781c28be667a15b31/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.430808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=593c4a93cb9a442a9d2f712d0942e950, user_id=9080d764164c443781c28be667a15b31, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.436849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2d1d1-3ec9-4446-8797-70e91e29cc09 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.439226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 948/1923] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:55 2022] PUT /identity/v3/projects/c54db5bc40e141fe91de819c80a28473/users/7cdbc4270cac4de69f73c654c6eb085d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:55.442983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.454850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 976/1924] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:55 2022] PUT /identity/v3/projects/593c4a93cb9a442a9d2f712d0942e950/users/9080d764164c443781c28be667a15b31/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:55.465488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.465784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.466112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.500540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] MFA Rules not processed for user `9080d764164c443781c28be667a15b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:55.501812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:55.508542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:55.565339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a50053-e67d-4306-9a3a-d1080c41700f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.620497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 977/1925] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:55.645017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.648425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:55.651644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.652318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.652600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.652876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.653576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.657132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.684026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 949/1926] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:25:55.688095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:55.706522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd548-c7ee-4ca9-9388-5fd44f60a61a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.716259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.718005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.719435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.720318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] PATH_INFO: `/v3/projects/c54db5bc40e141fe91de819c80a28473/users/7cdbc4270cac4de69f73c654c6eb085d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.723574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54db5bc40e141fe91de819c80a28473, user_id=7cdbc4270cac4de69f73c654c6eb085d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:55.727420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490779e0-75ce-40a7-896e-861ba3202171 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:55.744987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 950/1927] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:55 2022] PUT /identity/v3/projects/c54db5bc40e141fe91de819c80a28473/users/7cdbc4270cac4de69f73c654c6eb085d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:55.756253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.756999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.757628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.767189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183686dc-44a5-4cea-9a58-effd7bd3774e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:55.833052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] MFA Rules not processed for user `7cdbc4270cac4de69f73c654c6eb085d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:55.835075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:55.864071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:55.866380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:55.867045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:55.867336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:55.886216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:56.009820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:56.011590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 951/1928] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:56.016661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:56.028821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 978/1929] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:56.169854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:56.183661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6cc00-22cc-42a7-8756-b83b56990616 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:56.187588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:56.188664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.188664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.188852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.200867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:56.202870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f48622-efba-47ef-bfbd-bb11e4af2f24 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:56.220153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 952/1930] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:56.234412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:56.247800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:56.335927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95f8c3-3e84-49aa-b5a1-a731ff2318d3 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:56.340844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:56.341564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.341913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.342352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] PATH_INFO: `/v3/users/ebbac80e8ee34202bc28ee0f65e67ee9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.349876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbac80e8ee34202bc28ee0f65e67ee9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:56.352004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:56.412569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbac80e8ee34202bc28ee0f65e67ee9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.413717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbac80e8ee34202bc28ee0f65e67ee9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.414596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbac80e8ee34202bc28ee0f65e67ee9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.435562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbac80e8ee34202bc28ee0f65e67ee9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.446605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbac80e8ee34202bc28ee0f65e67ee9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.482635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 979/1931] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:56 2022] DELETE /identity/v3/users/ebbac80e8ee34202bc28ee0f65e67ee9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:56.488335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.488681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.489069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.514903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:56.516244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:56.595798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 953/1932] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:56.602530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:56.605278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:56.682862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9615-ac83-4812-b96f-84f60643bf82 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:56.687569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:56.688189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.688550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.688902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.701067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:56.704156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932665e-9419-464c-9d95-df608cfded38 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:56.716435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 980/1933] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:56.737049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:56.738452 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:56.750775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753e24-6b25-4d25-974d-c0ed271c4d7a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:56.755279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:56.755937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.756238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.756505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] PATH_INFO: `/v3/users/68bab262c80b49a882cabefcfe3ccd9e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.757698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68bab262c80b49a882cabefcfe3ccd9e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:56.759597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:56.808695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68bab262c80b49a882cabefcfe3ccd9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.809257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68bab262c80b49a882cabefcfe3ccd9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.809866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68bab262c80b49a882cabefcfe3ccd9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.824216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68bab262c80b49a882cabefcfe3ccd9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.831505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68bab262c80b49a882cabefcfe3ccd9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:56.846137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 954/1934] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:56 2022] DELETE /identity/v3/users/68bab262c80b49a882cabefcfe3ccd9e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:56.850525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:56.850814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:56.851102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:56.871030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:56.872270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:56.907163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:56.950831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 981/1935] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:56.960423 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:56.961989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:56.982848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:57.025319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d81d3-20bf-4b2c-874b-2d062cb21567 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.030392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.031074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.031357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.032115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.041112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.042739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.056237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 955/1936] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:57.071842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:57.074775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:57.108895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac4488-0bba-4f24-985f-0f03621db4c8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.114839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.115547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.115824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.116134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] PATH_INFO: `/v3/projects/5a3f698a181841799da576aa3e4b630b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.117303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3f698a181841799da576aa3e4b630b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.119273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.169640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3f698a181841799da576aa3e4b630b', 'request_id': 'req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.178215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3f698a181841799da576aa3e4b630b', 'request_id': 'req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.186711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3f698a181841799da576aa3e4b630b', 'request_id': 'req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.200645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] The token cache is being invalidate because project 5a3f698a181841799da576aa3e4b630b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:57.201375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.202217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.204286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 982/1937] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:57 2022] DELETE /identity/v3/projects/5a3f698a181841799da576aa3e4b630b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:57.208028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:57.218747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bab1b2-9ea5-4416-8159-4e844a036171 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.297130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.298052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.298335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.301491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] PATH_INFO: `/v3/users/9080d764164c443781c28be667a15b31` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.301491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9080d764164c443781c28be667a15b31)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.301916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.371050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9080d764164c443781c28be667a15b31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.371803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9080d764164c443781c28be667a15b31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.372526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9080d764164c443781c28be667a15b31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.385743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9080d764164c443781c28be667a15b31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.400834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9080d764164c443781c28be667a15b31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.428877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 956/1938] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:57 2022] DELETE /identity/v3/users/9080d764164c443781c28be667a15b31 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:57.449685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:57.462913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:57.465675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6f3a-85b7-4ff7-9bf3-5d117aa1b51c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.550775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.551446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.551721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.552017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] PATH_INFO: `/v3/projects/f11bb5290b2c4df6948fedffeb903104` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.553156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f11bb5290b2c4df6948fedffeb903104)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.555397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.607457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f11bb5290b2c4df6948fedffeb903104', 'request_id': 'req-1db28d72-4b97-4a88-be81-df14a6142cf9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.615245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:57.619114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f11bb5290b2c4df6948fedffeb903104', 'request_id': 'req-1db28d72-4b97-4a88-be81-df14a6142cf9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.635716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f11bb5290b2c4df6948fedffeb903104', 'request_id': 'req-1db28d72-4b97-4a88-be81-df14a6142cf9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.651685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] The token cache is being invalidate because project f11bb5290b2c4df6948fedffeb903104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:57.652280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.652975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.654711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 983/1939] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:57 2022] DELETE /identity/v3/projects/f11bb5290b2c4df6948fedffeb903104 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:57.658811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:57.660262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:57.673812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631f482-3737-46f7-aafd-7c2832c7cc21 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.712920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:57.737681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.738662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.738662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.738858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] PATH_INFO: `/v3/users/7cdbc4270cac4de69f73c654c6eb085d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.740087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdbc4270cac4de69f73c654c6eb085d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.742128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.762006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:57.768528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:57.785451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d72-4b97-4a88-be81-df14a6142cf9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:57.791401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:57.792057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.792382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.792621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] PATH_INFO: `/v3/projects/593c4a93cb9a442a9d2f712d0942e950` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.793978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=593c4a93cb9a442a9d2f712d0942e950)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:57.797044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:57.802319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdbc4270cac4de69f73c654c6eb085d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.803115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdbc4270cac4de69f73c654c6eb085d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.803730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdbc4270cac4de69f73c654c6eb085d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.817273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdbc4270cac4de69f73c654c6eb085d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.825096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08480712-1168-4fe3-b77b-100d533356b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdbc4270cac4de69f73c654c6eb085d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.841784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 957/1940] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:25:57 2022] DELETE /identity/v3/users/7cdbc4270cac4de69f73c654c6eb085d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:57.860490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '593c4a93cb9a442a9d2f712d0942e950', 'request_id': 'req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.866230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '593c4a93cb9a442a9d2f712d0942e950', 'request_id': 'req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.876227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '593c4a93cb9a442a9d2f712d0942e950', 'request_id': 'req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.890303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] The token cache is being invalidate because project 593c4a93cb9a442a9d2f712d0942e950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:57.890303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.891442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39201ea8-7d9b-4aa0-9fa8-d4f3cd6ad821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:57.892820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 984/1941] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:57 2022] DELETE /identity/v3/projects/593c4a93cb9a442a9d2f712d0942e950 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:57.898059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:57.899004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:57.899338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:57.925676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:57.927084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.018227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 958/1942] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:58.026145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205e331-146b-474b-b595-a411a72c7251 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.026338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205e331-146b-474b-b595-a411a72c7251 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.026619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205e331-146b-474b-b595-a411a72c7251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.052457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9205e331-146b-474b-b595-a411a72c7251 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.054156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9205e331-146b-474b-b595-a411a72c7251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.064203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 985/1943] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:58.068502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.078239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e4eae-d5e9-4c9e-afb5-ec851a5be95d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.083194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.083597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.083879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.084210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.084921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:58.085222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.086837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.099492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 959/1944] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:25:58.103946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205e331-146b-474b-b595-a411a72c7251 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.114778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205e331-146b-474b-b595-a411a72c7251 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.122114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.122680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.123027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.123372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.124586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.126206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.163999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 986/1945] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.168505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.168993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.171042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.182246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba652-f541-471a-a2ec-aca912e31483 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.239515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.262869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.263619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.263977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.265243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.265789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.267833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.284120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffd08a-5037-43ac-a1ca-fc9a7b74b0b3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.289476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.290315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.290609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.290905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] PATH_INFO: `/v3/projects/c54db5bc40e141fe91de819c80a28473` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.292126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54db5bc40e141fe91de819c80a28473)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.294221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.299062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 960/1946] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.303049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.312857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534827d9-0f56-49e5-8f52-95cf14ea8221 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.319463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.320115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.320390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.320724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.321471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.329165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.341510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 961/1947] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:58.346861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.349868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.357401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899acd51-a662-4855-a18c-8cade089c687 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.358162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54db5bc40e141fe91de819c80a28473', 'request_id': 'req-a503ffac-74ea-4eb9-80c9-ff4141c558e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:58.364090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54db5bc40e141fe91de819c80a28473', 'request_id': 'req-a503ffac-74ea-4eb9-80c9-ff4141c558e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:58.371464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54db5bc40e141fe91de819c80a28473', 'request_id': 'req-a503ffac-74ea-4eb9-80c9-ff4141c558e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:58.382920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] The token cache is being invalidate because project c54db5bc40e141fe91de819c80a28473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:25:58.383151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:58.383816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a503ffac-74ea-4eb9-80c9-ff4141c558e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:25:58.385425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 987/1948] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:25:58 2022] DELETE /identity/v3/projects/c54db5bc40e141fe91de819c80a28473 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:58.390806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9204-ebdd-41b7-b753-4ce1188c16b0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.391094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9204-ebdd-41b7-b753-4ce1188c16b0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.391376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9204-ebdd-41b7-b753-4ce1188c16b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.410743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f6be9204-ebdd-41b7-b753-4ce1188c16b0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.411970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6be9204-ebdd-41b7-b753-4ce1188c16b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.414261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.414878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.415154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.415425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.416135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.417715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.419830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 988/1949] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:58.427976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 962/1950] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:58.428931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.429217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.429497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.431721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.433402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.445752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a47f8-1651-461a-a26a-f8b53d350711 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.448857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.450024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.450580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.450991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.451269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.451540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] PATH_INFO: `/v3/projects/24f130ac2e214173a8078fa4f4cd67e3/users/e726e269568b43d39399d59f328cccd8/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.458398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 989/1951] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:58.462618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.468195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f130ac2e214173a8078fa4f4cd67e3, user_id=e726e269568b43d39399d59f328cccd8, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.470477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62949eed-1b4f-4eef-bec6-51494e77ea6a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.472290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d20074-7394-4060-96f9-aff62dd9266a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.477194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.477810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.478083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.478364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.479138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:25:58.479365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.481006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.485204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 963/1952] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:58 2022] PUT /identity/v3/projects/24f130ac2e214173a8078fa4f4cd67e3/users/e726e269568b43d39399d59f328cccd8/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:25:58.497114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 990/1953] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:25:58.497380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.497380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.497380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.498182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.510193 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc9b88-4770-4622-ae96-cb6ccad10e39 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.552174 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] MFA Rules not processed for user `e726e269568b43d39399d59f328cccd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.553502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.630641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.631889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.632274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.633018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.633656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.637295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.668071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 964/1954] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:58.669448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.683444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c375f8-65ad-408a-8986-e2a5627d8758 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.689718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 991/1955] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.692501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.738575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.782568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.783397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.783672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.784053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.784961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.787321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.791437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888eaec-6de1-468d-a673-561144a94bd2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.797246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.798179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.798461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.798727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.799397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.801564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.825035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 965/1956] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.826020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.839214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 992/1957] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.841797 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.856466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa103af-4fc5-4d25-b25d-6b4a6ed23040 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.876932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.917842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e457e-d0de-4eee-858e-f09fea49c3d6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:58.926083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.926396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.927100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.927100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.927611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.929795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.961394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:58.962030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.962314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.962587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.963618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:58.965870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:58.982311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 966/1958] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:25:58.986790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:58.996606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 993/1959] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:58.997386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07658387-3fb1-4f83-a4e6-09cf795d509b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.001203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.004062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.005429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.005708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.005977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.007141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.008779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.012146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4af67-70da-4319-9efd-06779d66a55d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.017997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.018625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.018907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.019183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.019933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.021876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.024133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 967/1960] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:59.027691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.036209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 994/1961] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:59.039488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee5189-4b5f-4e37-ad8d-8df8454d4dc8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.040766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.043735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.044535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.046275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.046711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] PATH_INFO: `/v3/projects/786062c97d71469786a33a38fe2a1013/users/a3f6dfe459f0463799e6a2a64b0715d4/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.053454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5140aa-e927-4e7f-a42a-80b850199b60 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.059515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.060329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.060623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.061126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] PATH_INFO: `/v3/projects/bfa9e26827f040138f562f84748f4063/users/7ed1d5851aed447e9f24a1f985553f3a/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.068698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786062c97d71469786a33a38fe2a1013, user_id=a3f6dfe459f0463799e6a2a64b0715d4, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.076479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.084694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa9e26827f040138f562f84748f4063, user_id=7ed1d5851aed447e9f24a1f985553f3a, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.089425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c1ee-7248-40b4-b609-e1448032d710 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.108824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 968/1962] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:59 2022] PUT /identity/v3/projects/786062c97d71469786a33a38fe2a1013/users/a3f6dfe459f0463799e6a2a64b0715d4/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:59.110282 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.119882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 995/1963] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:59 2022] PUT /identity/v3/projects/bfa9e26827f040138f562f84748f4063/users/7ed1d5851aed447e9f24a1f985553f3a/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:59.125938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.127224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.127224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.149626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:59.150847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:59.195066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8b0a-975e-4916-89ac-0297f73b9d75 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.216291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:59.282360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 996/1964] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:59.299348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.300967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:59.323124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.324005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.324481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.324757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.325472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.328502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.347305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 969/1965] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:25:59.352018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.373173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34796-fc86-48f6-90ae-b3dac9593d1b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.395312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.395312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.395312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.395312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] PATH_INFO: `/v3/projects/786062c97d71469786a33a38fe2a1013/users/a3f6dfe459f0463799e6a2a64b0715d4/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.395938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6a3b6-a4a7-46f5-ac4b-18045e09ab18 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.396149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786062c97d71469786a33a38fe2a1013, user_id=a3f6dfe459f0463799e6a2a64b0715d4, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.396242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.396242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.396242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.396242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.396543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16c691-4d07-480b-a3ac-8573ef57e9b2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.409997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.411825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.413684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 970/1966] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:25:59 2022] PUT /identity/v3/projects/786062c97d71469786a33a38fe2a1013/users/a3f6dfe459f0463799e6a2a64b0715d4/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:25:59.423861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 997/1967] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:59.429296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.429585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.429863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.461931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] MFA Rules not processed for user `a3f6dfe459f0463799e6a2a64b0715d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:59.463137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:59.550265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 971/1968] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:59.558727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:59.560197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:59.589433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:59.649242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd46-8132-47a8-95d3-64cbac87bb59 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:59.719209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:59.719942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:59.720362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:59.721026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:59.730825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:59.733093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:59.743104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 998/1969] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:00.240555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.250985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4a362-6f34-407f-b763-1ad5c6bc45b5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.265917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f6dfe459f0463799e6a2a64b0715d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786062c97d71469786a33a38fe2a1013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.266630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.266907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.267188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.268474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:00.268790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.270441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122f9e2-b8c0-4f3f-91f6-206a1bba0fd3 tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.294446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 972/1970] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:00.300420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.314893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ff3a-baeb-4da2-b3b1-e8337ba6db06 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.327704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.328455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.329416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.335046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f6dfe459f0463799e6a2a64b0715d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786062c97d71469786a33a38fe2a1013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.336004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.336189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.336657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.337481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.339702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.383196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] MFA Rules not processed for user `7ed1d5851aed447e9f24a1f985553f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:00.385162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:00.398879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 999/1971] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:00 2022] POST /identity/v3/projects => generated 352 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:00.403121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.468852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:00.478420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 973/1972] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:00.482791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.500491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0d440-a492-4eb9-b5cb-89dd64405c3f tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.506259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f6dfe459f0463799e6a2a64b0715d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786062c97d71469786a33a38fe2a1013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.507360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.507360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.507533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.508176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:00.508488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.510324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.526291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1000/1973] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:00.570349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.581132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c784-613b-42a8-8f76-3f65738e89cc tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.585776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f6dfe459f0463799e6a2a64b0715d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786062c97d71469786a33a38fe2a1013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.586420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.586554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a6b8a-de3d-44e3-811d-3d3914cf74db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.586700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.587028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] PATH_INFO: `/v3/projects/e6a290671ded4f50b723997fb206ac46` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.588243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e6a290671ded4f50b723997fb206ac46)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.590154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.644070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:00.648808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a290671ded4f50b723997fb206ac46', 'request_id': 'req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:00.656208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.657041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.657338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.657655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.657843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a290671ded4f50b723997fb206ac46', 'request_id': 'req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:00.658437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.660374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.670259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a290671ded4f50b723997fb206ac46', 'request_id': 'req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:00.683825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] The token cache is being invalidate because project e6a290671ded4f50b723997fb206ac46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:00.684505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:00.684819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:00.689343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1001/1974] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:00 2022] DELETE /identity/v3/projects/e6a290671ded4f50b723997fb206ac46 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:00.691516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 974/1975] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:00 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:00.695808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.697422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:00.803247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067c7-ab25-4ae8-9e9e-ce4980fd05ed tempest-VolumeQuotaClassesTest-208008854 tempest-VolumeQuotaClassesTest-208008854-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.901790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.901980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.902254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.903016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.903225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.905183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.953920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1002/1976] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:00 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:00.957980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.975256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6a5f-e1b4-4940-9e24-47d99fa41c89 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:00.980431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:00.981063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:00.981467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:00.981648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:00.982345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:00.984576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:00.988351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:00.990427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:00.999604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 975/1977] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:01.004439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:01.008890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a82d88-4f9d-4ccf-aa45-7a24ff410a0d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.015333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.016172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.016558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.016930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] PATH_INFO: `/v3/users/e726e269568b43d39399d59f328cccd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.017549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe7c6-84a1-41fa-9cbd-a06dacc0ac61 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.018664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e726e269568b43d39399d59f328cccd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.022108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.024014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.025227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.025516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.025793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] PATH_INFO: `/v3/projects/b48272602cb840dc8ddc45c516962048/users/a1d9100977e84a7d80218638a763e8a3/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.043460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48272602cb840dc8ddc45c516962048, user_id=a1d9100977e84a7d80218638a763e8a3, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.048487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.067519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 976/1978] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:01 2022] PUT /identity/v3/projects/b48272602cb840dc8ddc45c516962048/users/a1d9100977e84a7d80218638a763e8a3/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:01.072220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:01.124564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e726e269568b43d39399d59f328cccd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:01.126805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e726e269568b43d39399d59f328cccd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:01.127148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e726e269568b43d39399d59f328cccd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:01.151655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e726e269568b43d39399d59f328cccd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:01.153232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:01.169188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ebc4935-a51a-4fc5-aa7d-c25ffe15b3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e726e269568b43d39399d59f328cccd8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:01.172185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:01.186204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6843-8da8-4971-88a1-02cfac45fcd9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.196500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.197134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.197861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.197861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.198393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.200002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.205783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1003/1979] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:00 2022] DELETE /identity/v3/users/e726e269568b43d39399d59f328cccd8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:01.207266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.207549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.207819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.223351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 977/1980] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:01.225476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:01.243050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:01.243050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.247880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.257356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdb71b-43de-4ea7-9eab-8df2f2583d04 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.339343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1004/1981] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:01.350310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:01.352630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.364351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.365111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.365401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.365681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] PATH_INFO: `/v3/projects/b48272602cb840dc8ddc45c516962048/users/a1d9100977e84a7d80218638a763e8a3/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.372023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48272602cb840dc8ddc45c516962048, user_id=a1d9100977e84a7d80218638a763e8a3, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.377685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.399387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 978/1982] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:01 2022] PUT /identity/v3/projects/b48272602cb840dc8ddc45c516962048/users/a1d9100977e84a7d80218638a763e8a3/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:01.476161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ccaaf-79ed-4616-8ccf-edb45eaac571 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.588176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.592973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.593296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.593539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.656376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:01.671107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.674532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.691740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1005/1983] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:02.647132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:02.648970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:02.671195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70aabf-ed52-465b-91d1-51a60a09273f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:02.682664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:02.683334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:02.683621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:02.683980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] PATH_INFO: `/v3/projects/24f130ac2e214173a8078fa4f4cd67e3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:02.690974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f130ac2e214173a8078fa4f4cd67e3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:02.692952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:02.766627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:02.769031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:02.787632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18524f-2b3a-4d69-b969-3faa18844a02 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:02.798986 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f130ac2e214173a8078fa4f4cd67e3', 'request_id': 'req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:02.810467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f130ac2e214173a8078fa4f4cd67e3', 'request_id': 'req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:02.818692 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f130ac2e214173a8078fa4f4cd67e3', 'request_id': 'req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:02.843049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] The token cache is being invalidate because project 24f130ac2e214173a8078fa4f4cd67e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:02.843644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:02.844632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a25be8b7-d81b-4548-bc1a-ffbcdfbbbfb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:02.850184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 979/1984] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:02 2022] DELETE /identity/v3/projects/24f130ac2e214173a8078fa4f4cd67e3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:02.850184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:02.850184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:02.850184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:02.887537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] MFA Rules not processed for user `a1d9100977e84a7d80218638a763e8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:02.892116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:02.928239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:02.928935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:02.929218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:02.929494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] PATH_INFO: `/v3/users/fdaf221f93ba492db9a80ea58d5f82c9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:02.930775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdaf221f93ba492db9a80ea58d5f82c9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:02.932684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:02.971907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:02.992984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 980/1985] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:02.998503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:03.003898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.022438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54a650-2a5b-423b-a2aa-21bb06b035d8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:03.024591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdaf221f93ba492db9a80ea58d5f82c9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.025829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdaf221f93ba492db9a80ea58d5f82c9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.026617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdaf221f93ba492db9a80ea58d5f82c9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.028678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:03.029424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.030331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.030697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] PATH_INFO: `/v3/users/a3f6dfe459f0463799e6a2a64b0715d4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.032186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f6dfe459f0463799e6a2a64b0715d4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:03.034401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:03.048437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdaf221f93ba492db9a80ea58d5f82c9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.072913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdaf221f93ba492db9a80ea58d5f82c9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.099475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1006/1986] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:02 2022] DELETE /identity/v3/users/fdaf221f93ba492db9a80ea58d5f82c9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:03.100865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:03.102719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.134026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f6dfe459f0463799e6a2a64b0715d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.135518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f6dfe459f0463799e6a2a64b0715d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.136698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f6dfe459f0463799e6a2a64b0715d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.164642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f6dfe459f0463799e6a2a64b0715d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.181971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-38987daa-d7fa-4865-bd64-a5a9b0906dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f6dfe459f0463799e6a2a64b0715d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:03.206129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 981/1987] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:02 2022] DELETE /identity/v3/users/a3f6dfe459f0463799e6a2a64b0715d4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:03.206348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633cc92-45c6-41bc-a203-b09675572569 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:03.206451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.206451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.206451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.230891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:03.232169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.261837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:03.318433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:03.319203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.332109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.332109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.332109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.347319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 982/1988] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:03.358450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:03.360216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.400311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:03.401912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:03.415715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1007/1989] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:03.424697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c0319-bd23-46c4-9dfb-559b83ac9e22 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:03.429169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:03.430110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.430416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.430760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.442649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:03.444212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:03.456940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 983/1990] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:03.863707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:03.865942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.895323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42712-7a17-4bb4-adb0-47d5fadbdf65 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:03.910119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:03.911053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.911556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.912617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] PATH_INFO: `/v3/projects/97f388eafa6544a7b966af69910652c4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.914416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f388eafa6544a7b966af69910652c4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:03.916921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:04.049362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f388eafa6544a7b966af69910652c4', 'request_id': 'req-ed085115-30e6-4510-8b81-2146c295480d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.057648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f388eafa6544a7b966af69910652c4', 'request_id': 'req-ed085115-30e6-4510-8b81-2146c295480d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.069844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f388eafa6544a7b966af69910652c4', 'request_id': 'req-ed085115-30e6-4510-8b81-2146c295480d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.090421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] The token cache is being invalidate because project 97f388eafa6544a7b966af69910652c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:04.091031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.094095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.103876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.104669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1008/1991] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:03 2022] DELETE /identity/v3/projects/97f388eafa6544a7b966af69910652c4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:04.130544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbeddf-bcfb-4d4b-9912-966df73c748a service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.141840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:04.209173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:04.209833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.210248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.210531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] PATH_INFO: `/v3/users/a9ccd215fcdd4879b634efa1db348bc2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.211729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ccd215fcdd4879b634efa1db348bc2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:04.213711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:04.307103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ccd215fcdd4879b634efa1db348bc2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.308246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ccd215fcdd4879b634efa1db348bc2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.309192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ccd215fcdd4879b634efa1db348bc2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.328474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.330091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:04.340954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ccd215fcdd4879b634efa1db348bc2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.351701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed085115-30e6-4510-8b81-2146c295480d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.363229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d1138a6a-a83b-4970-bb74-146495205a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ccd215fcdd4879b634efa1db348bc2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.375119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:04.375789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.376244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.376535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] PATH_INFO: `/v3/projects/786062c97d71469786a33a38fe2a1013` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.378010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786062c97d71469786a33a38fe2a1013)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:04.380022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:04.393621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 984/1992] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:04 2022] DELETE /identity/v3/users/a9ccd215fcdd4879b634efa1db348bc2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:04.454416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786062c97d71469786a33a38fe2a1013', 'request_id': 'req-72bbaa02-6f69-4bbf-be4a-a55067a08747'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.460000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:04.461476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786062c97d71469786a33a38fe2a1013', 'request_id': 'req-72bbaa02-6f69-4bbf-be4a-a55067a08747'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.472930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786062c97d71469786a33a38fe2a1013', 'request_id': 'req-72bbaa02-6f69-4bbf-be4a-a55067a08747'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.497828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] The token cache is being invalidate because project 786062c97d71469786a33a38fe2a1013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:04.498128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.503691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72bbaa02-6f69-4bbf-be4a-a55067a08747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:04.510152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1009/1993] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:04 2022] DELETE /identity/v3/projects/786062c97d71469786a33a38fe2a1013 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:04.512686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.513104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.513363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.557172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:04.560099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:04.728350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.728685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.729174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.731678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 985/1994] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:04.760929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:04.765311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:04.782031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1010/1995] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:04.783390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.804890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bec24-75e8-4bf3-b3a2-9e2020d4f2a7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.812305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:04.813162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.813660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.814123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.815002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:04.815539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:04.817387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:04.836858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 986/1996] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:04.838649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.852385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcfd6d-c69c-4f59-b35d-49f483b968c1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.859152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:04.860050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.860526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.860965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.861783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:04.863906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:04.901344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1011/1997] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:04 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:04.903528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.919805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2fc3-dc30-4a9e-8623-f06d83ed619a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.937879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:05.023582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.025345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.025870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.026350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.027293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.030425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.064129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 987/1998] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:04 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:05.068174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.080208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b321-c428-416f-a80d-8ae03567535e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.104297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.105514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.105935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.106310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.108966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.111164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:05.118194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.121476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.149747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1012/1999] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:05.152244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.170757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0cf49-c72e-48ec-b6be-3835947234c2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.177954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.178736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.179095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.179447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.180270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.182230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.224125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1013/2000] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:05.227256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.239990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf02f-30f1-41be-a684-9795564505a7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.251232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.251232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.251232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.251232 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.256219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c316c2-cd1a-464d-87a5-4dd69d2865d8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.257095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:05.266838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.268482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.268842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.269187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] PATH_INFO: `/v3/projects/4c5b61df1aae42f39ca52348c4153325/users/74892946f15441ce9a52850b18a4844a/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.292128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5b61df1aae42f39ca52348c4153325, user_id=74892946f15441ce9a52850b18a4844a, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.296412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c35a-7d6f-482d-b5f3-224c6aba69e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.334405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1014/2001] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:05 2022] PUT /identity/v3/projects/4c5b61df1aae42f39ca52348c4153325/users/74892946f15441ce9a52850b18a4844a/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:05.346310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.346677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.347137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.390446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] MFA Rules not processed for user `74892946f15441ce9a52850b18a4844a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:05.392575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:05.417796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.421464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.445198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 988/2002] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:05.478690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1015/2003] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:05.479881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.492843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe922c-2238-445a-99cf-8bf8d3dd2105 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.572263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:05.574889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.576325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.576924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.577348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.578632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.583164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.626498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 989/2004] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:05 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:05.630511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.643794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe071ad-3b89-4794-8a35-90bec252d41a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.747790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.748594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.751196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.752914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.754061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.756620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.769035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:05.800019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1016/2005] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:05 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:05.801221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.816470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e2a1-b3b4-48de-9361-fd7d8f7def69 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.824755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.825435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.825719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.827179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.827467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.829091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.857698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 990/2006] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:05.859022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.869881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d73c-e94b-423a-940f-cd00dd7aea8e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.874965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.875676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.876054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.877009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.877924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.879587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbf928-007c-4735-ace8-e32ede0f4bd0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.891410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1017/2007] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:05.896248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:05.913269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb23da-5317-46f2-92eb-4c44c8546df0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:05.918482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:05.919176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.919460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.919739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] PATH_INFO: `/v3/projects/384e7fb63a814787a61ea4eea9f349e1/users/cdea3d66080644789d3a0b532b109936/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.953769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384e7fb63a814787a61ea4eea9f349e1, user_id=cdea3d66080644789d3a0b532b109936, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.956782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.986654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 991/2008] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:05 2022] PUT /identity/v3/projects/384e7fb63a814787a61ea4eea9f349e1/users/cdea3d66080644789d3a0b532b109936/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:06.000079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.000433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.000722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.042512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] MFA Rules not processed for user `cdea3d66080644789d3a0b532b109936`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:06.043860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.133930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1018/2009] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:06.139827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.258534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bc5d6-07b9-4bf9-b820-345b0f6500a0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.267234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.267234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.267234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.267234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.267739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.269053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.271706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.275182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.300950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4be5cf-2974-438c-9ca7-d061e8186f6f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.308517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 992/2010] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:06.313092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.333585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.334254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.334535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.334808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] PATH_INFO: `/v3/projects/dd4ad84b0048403697cf20d6b1646235` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.336077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4ad84b0048403697cf20d6b1646235)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.338008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.414182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4ad84b0048403697cf20d6b1646235', 'request_id': 'req-3809ac44-8347-4aca-aa13-cbdabcd7444d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:06.423299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4ad84b0048403697cf20d6b1646235', 'request_id': 'req-3809ac44-8347-4aca-aa13-cbdabcd7444d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:06.434860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4ad84b0048403697cf20d6b1646235', 'request_id': 'req-3809ac44-8347-4aca-aa13-cbdabcd7444d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:06.435523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf210d-4051-4bd9-83da-76af3a8dae0d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.455979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] The token cache is being invalidate because project dd4ad84b0048403697cf20d6b1646235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:06.456279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:06.456965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3809ac44-8347-4aca-aa13-cbdabcd7444d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:06.462019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1019/2011] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:06 2022] DELETE /identity/v3/projects/dd4ad84b0048403697cf20d6b1646235 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:06.463884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5654-8dd8-4fc0-9786-ccc4c60ea2cd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.464215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5654-8dd8-4fc0-9786-ccc4c60ea2cd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.464634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5654-8dd8-4fc0-9786-ccc4c60ea2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.493680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:06.496678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1aae5654-8dd8-4fc0-9786-ccc4c60ea2cd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:06.498398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1aae5654-8dd8-4fc0-9786-ccc4c60ea2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.533021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.533908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.534424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.535017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.535870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.539707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.565832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:06.572422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 993/2012] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/users => generated 295 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:06.573606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.575666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.583799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1020/2013] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:06.592750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.593282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.593634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.600779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe019-bec5-4bc0-9e43-4393e4649946 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.606167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.608328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.608876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.609347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.610330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.612415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.616726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:06.618035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.625638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 994/2014] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:06.630196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.635271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1021/2015] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:06.641016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.644149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f774-fb81-44e7-b0f6-5396839f00ab admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.650234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.650922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.651205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.651483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] PATH_INFO: `/v3/projects/f6696ebf6503462aa5a585ff5d006936/users/3dd22960b26447f8b1fc006df507f6c0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.656471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fa2bc-6fda-427c-8dab-4e95342b3bbf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.665820 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.666794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.667422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.667915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.669455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:06.669993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.672333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.687410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6696ebf6503462aa5a585ff5d006936, user_id=3dd22960b26447f8b1fc006df507f6c0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.689025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1022/2016] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:06.690140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.693505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.707932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a598a-39f9-4a5f-b7f6-4100d6c014bb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.713422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 995/2017] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:06 2022] PUT /identity/v3/projects/f6696ebf6503462aa5a585ff5d006936/users/3dd22960b26447f8b1fc006df507f6c0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:06.714344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.785222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.786065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.786444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.786796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.787657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.789947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.800841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9937c4-5bbb-459c-b7e6-fa42efbf723b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.806150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.806866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.807221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.807574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.808320 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.809925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.821805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 996/2018] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:06.825801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.838350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1023/2019] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:06.841019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.841898 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62f42a-d1e7-41ab-976d-b07aee2a04d1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.927211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468f64c-29aa-4d78-a681-11b6c0f70629 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.935014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.935648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.936536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.936811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.937480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.939538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.947157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.955686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.956144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.956520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] PATH_INFO: `/v3/projects/f6696ebf6503462aa5a585ff5d006936/users/3dd22960b26447f8b1fc006df507f6c0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.959765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6696ebf6503462aa5a585ff5d006936, user_id=3dd22960b26447f8b1fc006df507f6c0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.962709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aab576-0cd3-4068-b5d7-382570a4d5ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.980531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 997/2020] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:06 2022] PUT /identity/v3/projects/f6696ebf6503462aa5a585ff5d006936/users/3dd22960b26447f8b1fc006df507f6c0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:06.991480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.992295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1024/2021] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:06.992688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.992978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.996164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.014190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcd04-29b8-4831-a995-161ac666226a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.034302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] MFA Rules not processed for user `3dd22960b26447f8b1fc006df507f6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:07.035850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:07.058571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:07.093313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:07.117939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.120324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.120875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.121325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.122301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.125860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.145104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1025/2022] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:07.152648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.167745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1f5f6-200e-4334-95e3-afd9016b3a10 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.174710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.176216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 998/2023] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:07.176633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.177117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.177537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.178902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.188245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.188431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.190442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:07.205902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1026/2024] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:07.210303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.228431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af69bfa-0af8-4970-8baf-d981877f3a0f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.256735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.257441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.257743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.258235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] PATH_INFO: `/v3/projects/842ddcf8a9ab41b1a02e55aaf171c560/users/5f11821ed33244aabfee40bab9c4cc50/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.277477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28b8c-f914-495a-adfd-685c5bd07976 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.284135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.285009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.285668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.286321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.304704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.306465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faff92-f2c8-430e-8b59-bc973b3ccb6e service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.344644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=842ddcf8a9ab41b1a02e55aaf171c560, user_id=5f11821ed33244aabfee40bab9c4cc50, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.348009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 999/2025] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:07.361754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.391567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1027/2026] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:07 2022] PUT /identity/v3/projects/842ddcf8a9ab41b1a02e55aaf171c560/users/5f11821ed33244aabfee40bab9c4cc50/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:07.406607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.406909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.407369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.431217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.494953 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] MFA Rules not processed for user `5f11821ed33244aabfee40bab9c4cc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:07.497687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:07.529767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa98a0-8e21-46b3-91ce-d2eb94376f94 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.543764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.544587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.544876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.545305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.547096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:07.691927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.693596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.706927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1000/2027] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:07.711311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1028/2028] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:07.745785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.770885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5257fd-dd77-4834-9bd2-36c1159986a7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.776602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.778162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.778688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.779166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.794483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.802992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fb6e5-865c-409a-bfbb-b8f7b8ebd590 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.819631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1001/2029] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:07.844668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:07.940274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea0f9-707b-4ed8-bd44-eef73e2de569 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:07.945171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:07.945877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.946174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.946453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] PATH_INFO: `/v3/users/5f11821ed33244aabfee40bab9c4cc50` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.947753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f11821ed33244aabfee40bab9c4cc50)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:07.949754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:07.952833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:08.009095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f11821ed33244aabfee40bab9c4cc50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.010394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f11821ed33244aabfee40bab9c4cc50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.010841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f11821ed33244aabfee40bab9c4cc50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.033777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f11821ed33244aabfee40bab9c4cc50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.049435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f11821ed33244aabfee40bab9c4cc50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.067836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1029/2030] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:07 2022] DELETE /identity/v3/users/5f11821ed33244aabfee40bab9c4cc50 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:08.072035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.072396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.072756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.106208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:08.107438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.131878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:08.178270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1002/2031] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:08.189404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:08.191000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.257495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d150ce5-99d2-43c5-8895-647afedb071f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:08.261187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:08.261802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.262078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.262347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.272369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:08.273804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07185b-13f9-4338-98f4-0108fc4d154b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:08.284864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1030/2032] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:08.597308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:08.599179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.615858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fab5a-be69-4a65-b04a-74fa9a3329fd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:08.621497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:08.622284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.622652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.624760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] PATH_INFO: `/v3/projects/842ddcf8a9ab41b1a02e55aaf171c560` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.626226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=842ddcf8a9ab41b1a02e55aaf171c560)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:08.628315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:08.690011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842ddcf8a9ab41b1a02e55aaf171c560', 'request_id': 'req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.698158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842ddcf8a9ab41b1a02e55aaf171c560', 'request_id': 'req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.711632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842ddcf8a9ab41b1a02e55aaf171c560', 'request_id': 'req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.745841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] The token cache is being invalidate because project 842ddcf8a9ab41b1a02e55aaf171c560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:08.746133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.746952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f0c519f3-3fff-4478-84ee-794d4e5a2a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:08.752052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1003/2033] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:08 2022] DELETE /identity/v3/projects/842ddcf8a9ab41b1a02e55aaf171c560 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:08.753435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.753701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.753960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.777891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:08.780191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.871730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1031/2034] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:08.879642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.879925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.880234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.900266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:08.901479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.910326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1004/2035] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:08.914539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:08.924801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666a19-54a3-4b91-a460-fbf0ae9059bf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:08.929532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:08.930131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.930406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.930727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.931418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:08.931716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:08.933403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:08.944740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1032/2036] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:08.948955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:08.958593 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4e610-e0e9-4ec2-9bf7-2a501c958bbd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:08.963245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:08.963847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.964206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.964442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.965101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:08.966988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:08.993489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1005/2037] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:08 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:08.997559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.008240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135714bc-3b9d-446d-a678-efa7678b5e8a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.016623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:09.066069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.066684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.066967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.067244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.067948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:09.069891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:09.090836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1033/2038] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:08 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:09.095345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.105753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a72ab-a01b-444e-a281-c256a77b9d6b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.110385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.111000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.111281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.111557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.112288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:09.113878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:09.124434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1006/2039] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:09.128685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.138653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6481f4-7ea7-47cb-a7b7-8afdcdc2c00d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.144032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.144708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.144991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.145261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.145948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:09.147575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79d0ba-b6a4-426b-aa8b-26c8f4bf7fa8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:09.158584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1034/2040] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:09.162909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.173615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6d7db-ede4-4d1f-ba00-c6a15e05aa05 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.179134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.179863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.180135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.180415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] PATH_INFO: `/v3/projects/fcc945cdc83549c3833b0e39a9496e26/users/1a7627957f0c489ebf27dcab206542ef/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.197840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc945cdc83549c3833b0e39a9496e26, user_id=1a7627957f0c489ebf27dcab206542ef, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:09.200688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:09.216322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1007/2041] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:09 2022] PUT /identity/v3/projects/fcc945cdc83549c3833b0e39a9496e26/users/1a7627957f0c489ebf27dcab206542ef/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:09.222173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622c13-b970-4044-8847-236775b8b413 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.222465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622c13-b970-4044-8847-236775b8b413 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.222743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622c13-b970-4044-8847-236775b8b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.242095 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e4622c13-b970-4044-8847-236775b8b413 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:09.243322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e4622c13-b970-4044-8847-236775b8b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:09.311307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1035/2042] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:09.322487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.324271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:09.382580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:09.388773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4041-bcbe-4902-a6ae-caa3e05f400f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.397032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.397672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.398096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.398371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.409349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:09.410784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:09.423147 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1008/2043] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:10.263607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.264167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.264167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.298993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] MFA Rules not processed for user `1a7627957f0c489ebf27dcab206542ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:10.300242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:10.369230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1036/2044] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:10.380843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:10.382246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:10.462718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d947097-2439-4119-b373-a740be5a7acf service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:10.466845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:10.467476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.468125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.468907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.480680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:10.482248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3c3a9-5f06-4209-876d-d8f0e4c64982 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:10.506146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1009/2045] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:10.765968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:10.767395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:10.783076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c45af2-959c-44e0-ab70-cfd9dec793ac None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:10.789170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:10.790237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.790602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.791581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] PATH_INFO: `/v3/users/74892946f15441ce9a52850b18a4844a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.793120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74892946f15441ce9a52850b18a4844a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:10.795465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:10.811246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:10.863319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74892946f15441ce9a52850b18a4844a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:10.864167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74892946f15441ce9a52850b18a4844a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:10.864910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74892946f15441ce9a52850b18a4844a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:10.883752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74892946f15441ce9a52850b18a4844a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:10.895544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74892946f15441ce9a52850b18a4844a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:10.910626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1037/2046] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:10 2022] DELETE /identity/v3/users/74892946f15441ce9a52850b18a4844a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:10.915315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.915654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.916034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.940903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:10.942099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.004556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1010/2047] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:11.014822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:11.023617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.088670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d85fb-c9bc-4b78-9bf1-df4df5961601 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:11.092494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:11.093108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:11.093406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:11.093677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:11.103461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:11.104943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:11.117612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1038/2048] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:11.483914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:11.485337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.502258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b051-fca2-4361-be14-1d412cea8d5f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:11.507637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:11.508323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:11.508595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:11.509041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] PATH_INFO: `/v3/projects/4c5b61df1aae42f39ca52348c4153325` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:11.511132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5b61df1aae42f39ca52348c4153325)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:11.513454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:11.561776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5b61df1aae42f39ca52348c4153325', 'request_id': 'req-bad27f43-3833-4aba-a950-9f98dc739f8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.567720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5b61df1aae42f39ca52348c4153325', 'request_id': 'req-bad27f43-3833-4aba-a950-9f98dc739f8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.573814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5b61df1aae42f39ca52348c4153325', 'request_id': 'req-bad27f43-3833-4aba-a950-9f98dc739f8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.586203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] The token cache is being invalidate because project 4c5b61df1aae42f39ca52348c4153325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:11.586488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.587089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bad27f43-3833-4aba-a950-9f98dc739f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.588613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1011/2049] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:11 2022] DELETE /identity/v3/projects/4c5b61df1aae42f39ca52348c4153325 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:11.592554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:11.601959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea4ec-50c3-4724-bf3a-09916796aec8 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:11.656793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:11.657374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:11.657654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:11.657928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] PATH_INFO: `/v3/users/cdea3d66080644789d3a0b532b109936` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:11.659152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdea3d66080644789d3a0b532b109936)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:11.661194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:11.699378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:11.708905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdea3d66080644789d3a0b532b109936'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.709830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdea3d66080644789d3a0b532b109936'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.710702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdea3d66080644789d3a0b532b109936'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.725285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdea3d66080644789d3a0b532b109936'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.732529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdea3d66080644789d3a0b532b109936'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:11.745747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1039/2050] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:11 2022] DELETE /identity/v3/users/cdea3d66080644789d3a0b532b109936 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:11.965239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:11.965792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:11.966076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:11.987451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] MFA Rules not processed for user `1a7627957f0c489ebf27dcab206542ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:11.988703 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.998219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:12.048283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1012/2051] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:12.050212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:12.051911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:12.133790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a5ed7-4d53-462f-94c4-66ceffc64340 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:12.138908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:12.139615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:12.139904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:12.140261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] PATH_INFO: `/v3/projects/384e7fb63a814787a61ea4eea9f349e1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:12.141429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=384e7fb63a814787a61ea4eea9f349e1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:12.143346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:12.191981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384e7fb63a814787a61ea4eea9f349e1', 'request_id': 'req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.197500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384e7fb63a814787a61ea4eea9f349e1', 'request_id': 'req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.205195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384e7fb63a814787a61ea4eea9f349e1', 'request_id': 'req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.216942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] The token cache is being invalidate because project 384e7fb63a814787a61ea4eea9f349e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:12.217207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.217847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.219947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1040/2052] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:12 2022] DELETE /identity/v3/projects/384e7fb63a814787a61ea4eea9f349e1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:12.223864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:12.235445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35104-bd37-47dc-a123-270e63d135ed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:12.294083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:12.294745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:12.295030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:12.295310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] PATH_INFO: `/v3/users/3dd22960b26447f8b1fc006df507f6c0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:12.296610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd22960b26447f8b1fc006df507f6c0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:12.298811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:12.347776 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd22960b26447f8b1fc006df507f6c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.348496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd22960b26447f8b1fc006df507f6c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.349111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd22960b26447f8b1fc006df507f6c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.361982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd22960b26447f8b1fc006df507f6c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.369315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f76d93b1-ba0a-4fc0-8859-63ae45f8fa72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd22960b26447f8b1fc006df507f6c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:12.388444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1013/2053] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:12 2022] DELETE /identity/v3/users/3dd22960b26447f8b1fc006df507f6c0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:12.835192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:12.837365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:12.866656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b122fe-8eff-4a5e-abcc-f3f9535fbe3e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:12.988048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:13.009048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.009716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.010164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.010596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] PATH_INFO: `/v3/projects/f6696ebf6503462aa5a585ff5d006936` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.011998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6696ebf6503462aa5a585ff5d006936)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.014295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.066010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6696ebf6503462aa5a585ff5d006936', 'request_id': 'req-cfb2bcea-52b1-4608-b92d-c8378f248ecc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:13.071238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6696ebf6503462aa5a585ff5d006936', 'request_id': 'req-cfb2bcea-52b1-4608-b92d-c8378f248ecc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:13.080775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6696ebf6503462aa5a585ff5d006936', 'request_id': 'req-cfb2bcea-52b1-4608-b92d-c8378f248ecc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:13.093985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] The token cache is being invalidate because project f6696ebf6503462aa5a585ff5d006936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:13.094304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:13.094954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cfb2bcea-52b1-4608-b92d-c8378f248ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:13.101140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1041/2054] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:12 2022] DELETE /identity/v3/projects/f6696ebf6503462aa5a585ff5d006936 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:13.102625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.103096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.103204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.123516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:13.125388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.162029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:13.191653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1014/2055] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:13.200191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.200488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.200766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.220498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:13.221597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.229947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1042/2056] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:13.234731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.244639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375008f-3fcb-410d-8bae-962a9504e574 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.249300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.250500 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.250817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.251566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.251783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:13.252132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.253720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.269129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1015/2057] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:26:13.270159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.281040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8ce38-72fa-4844-9221-1531bef9fa80 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.285971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.286633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.286929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.287203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.287871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.289883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.321864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1043/2058] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:13 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:13.326047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.336926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5666d18-0068-4ece-ada6-f332e41bcd58 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.399295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.399946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.400271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.400524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.401190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.403135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.429031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1016/2059] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:13 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:13.430195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.441037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bacbf7-8dfc-425c-9b54-1f50660096d9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.445654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.446254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.446532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.446802 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.447487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.449092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.458769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1044/2060] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:13.462649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.472957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10700-c6ae-4e5c-983d-c3e6c422415e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.477425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.478021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.478300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.478573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.479263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.480863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257d09c-dbc0-470f-a07c-7a32ac69aca4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.493258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1017/2061] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:13.497432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.507722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74a26a-2dfd-4dfa-bfa6-539acbfe66e3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.513010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.513682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.513962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.514236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] PATH_INFO: `/v3/projects/e87994ec0444411f9866120d041cc50a/users/0e434f8b9b1d48f3bd1a2071323ee217/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.529554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87994ec0444411f9866120d041cc50a, user_id=0e434f8b9b1d48f3bd1a2071323ee217, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.532959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.548966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1045/2062] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:13 2022] PUT /identity/v3/projects/e87994ec0444411f9866120d041cc50a/users/0e434f8b9b1d48f3bd1a2071323ee217/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:13.560202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c357e-d744-4cd4-a634-a6c7d5455ace None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.560384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c357e-d744-4cd4-a634-a6c7d5455ace None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.561066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c357e-d744-4cd4-a634-a6c7d5455ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.598356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9b2c357e-d744-4cd4-a634-a6c7d5455ace None None] MFA Rules not processed for user `0e434f8b9b1d48f3bd1a2071323ee217`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:13.599575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9b2c357e-d744-4cd4-a634-a6c7d5455ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.665715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1018/2063] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:13.677786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:13.679479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.759458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a872f58-9512-4785-8077-5b8f16f10319 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.763693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.764355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.764638 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.764916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.771980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:13.776707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.778884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca390de9-cf87-46e8-9fa2-3b46834e012b service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.793460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1046/2064] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:19.613228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:19.613799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:19.613799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:19.615321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1019/2065] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 10:26:19.654265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:19.654814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:19.654938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:19.677788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] MFA Rules not processed for user `47da679abbac42a1b3244ff9acafd577`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:19.679032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:19.763711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1047/2066] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:19.777227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:19.779037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:19.829795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfa196-c30c-4be2-8d6f-c01fd0961362 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:19.841611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:19.842669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:19.843426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:19.844059 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:19.884862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:19.886471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:19.920835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1020/2067] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:20.398973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:20.400177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:20.497730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e2a2c-d945-4e63-879a-0f20e487087d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:20.502995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:20.503845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:20.504166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:20.504446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] PATH_INFO: `/v3/users/7ed1d5851aed447e9f24a1f985553f3a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:20.505632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed1d5851aed447e9f24a1f985553f3a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:20.507864 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:20.569283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed1d5851aed447e9f24a1f985553f3a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:20.570054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed1d5851aed447e9f24a1f985553f3a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:20.570662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed1d5851aed447e9f24a1f985553f3a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:20.586034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed1d5851aed447e9f24a1f985553f3a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:20.598667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed1d5851aed447e9f24a1f985553f3a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:20.616006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1048/2068] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:20 2022] DELETE /identity/v3/users/7ed1d5851aed447e9f24a1f985553f3a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:21.016328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:21.022648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474cf8-74b4-4455-b0bb-161e9422367b service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:21.067220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:21.134740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:21.135523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:21.135879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:21.136278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] PATH_INFO: `/v3/projects/bfa9e26827f040138f562f84748f4063` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:21.137658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa9e26827f040138f562f84748f4063)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:21.139688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:21.225552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa9e26827f040138f562f84748f4063', 'request_id': 'req-0e425220-72d0-4ee9-83e3-ee07f55e871f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.235128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa9e26827f040138f562f84748f4063', 'request_id': 'req-0e425220-72d0-4ee9-83e3-ee07f55e871f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.247185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa9e26827f040138f562f84748f4063', 'request_id': 'req-0e425220-72d0-4ee9-83e3-ee07f55e871f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.265431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] The token cache is being invalidate because project bfa9e26827f040138f562f84748f4063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:21.265727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.268647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.269900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1021/2069] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:21 2022] DELETE /identity/v3/projects/bfa9e26827f040138f562f84748f4063 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:21.273905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:21.275488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:21.290439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f4ed-6c94-4772-aceb-e227c4dbffef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:21.296090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:21.364524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:21.365266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:21.365552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:21.365812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] PATH_INFO: `/v3/users/a1d9100977e84a7d80218638a763e8a3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:21.366989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d9100977e84a7d80218638a763e8a3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:21.369421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:21.431842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d9100977e84a7d80218638a763e8a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.432859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d9100977e84a7d80218638a763e8a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.433781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d9100977e84a7d80218638a763e8a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.448319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d9100977e84a7d80218638a763e8a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.461068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e550781-ac30-4e08-a843-35783a2656b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d9100977e84a7d80218638a763e8a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.476642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1049/2070] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:21 2022] DELETE /identity/v3/users/a1d9100977e84a7d80218638a763e8a3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:21.679759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:21.681304 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:21.717832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425220-72d0-4ee9-83e3-ee07f55e871f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:21.784494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:21.785167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:21.785451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:21.785732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] PATH_INFO: `/v3/projects/b48272602cb840dc8ddc45c516962048` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:21.786905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48272602cb840dc8ddc45c516962048)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:21.788943 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:21.846422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48272602cb840dc8ddc45c516962048', 'request_id': 'req-c738f065-f0fe-47de-8ef2-1367a73efbc3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.850798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:21.853924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48272602cb840dc8ddc45c516962048', 'request_id': 'req-c738f065-f0fe-47de-8ef2-1367a73efbc3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.863051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48272602cb840dc8ddc45c516962048', 'request_id': 'req-c738f065-f0fe-47de-8ef2-1367a73efbc3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.893237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] The token cache is being invalidate because project b48272602cb840dc8ddc45c516962048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:21.893596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.894357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c738f065-f0fe-47de-8ef2-1367a73efbc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:21.908133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1022/2071] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:21 2022] DELETE /identity/v3/projects/b48272602cb840dc8ddc45c516962048 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:21.918182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:21.918863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:21.918863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:21.943315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:21.944642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:22.026160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1050/2072] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:22.033985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.034514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.034514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.058297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:22.059636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:22.074828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1023/2073] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:22.079730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.089706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b30cb8-6fc3-41c1-8e7b-7a083b9d5460 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.094503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.095117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.095397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.095669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.096425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:22.096728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.098318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.115161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1051/2074] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:22.116360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.126416 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bd473-f7bb-4930-8eba-765e44e0ee2c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.131300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.131918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.132230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.132511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.133180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.135074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.168086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1024/2075] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:22.169275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.180203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633b3e-386e-489f-88e8-af77b30bc86f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.193774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:22.245527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.246121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.246403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.246677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.247347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.249394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.279584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1052/2076] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:22.283864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.295725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12e-dced-49b1-90a6-1f886495afd8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.301596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.302235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.302519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.302794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.303492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.305158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.320302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1025/2077] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:22.321559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.333246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d45c-7fc2-4c4b-9627-1ca104a07f71 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.338061 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.338711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.338996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.339282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.340025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.341620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ddae-1a38-47e8-91af-bedd1bb3a103 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.357463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1053/2078] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:22.362093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.372962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe0c6e-da22-45b3-ac85-76bdc7b7152e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.380392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.381077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.382300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.382300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] PATH_INFO: `/v3/projects/adb3fb2886d4402bb0696bc84cdfe1c1/users/0bb1051fa32a40a68ddc30b392a97323/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.406519 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb3fb2886d4402bb0696bc84cdfe1c1, user_id=0bb1051fa32a40a68ddc30b392a97323, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.409053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.438897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1026/2079] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:22 2022] PUT /identity/v3/projects/adb3fb2886d4402bb0696bc84cdfe1c1/users/0bb1051fa32a40a68ddc30b392a97323/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:22.449191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.449808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.450278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.487207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] MFA Rules not processed for user `0bb1051fa32a40a68ddc30b392a97323`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:22.488482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:22.568773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1054/2080] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:22.573626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.662397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2d57a-4491-4347-90a0-9580c9d17439 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.667024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.667665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.667945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.668266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.668932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.670833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.704718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1027/2081] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:22.709020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.722156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9f5de-f772-4232-b06a-783bbcf36ea4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.809741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.810691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.811150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.812721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.813521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.815576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.854756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1055/2082] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:22.858068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.872591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed6d0-5ff7-4811-9e9d-92eeb590c0da admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.874808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.876772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:22.884706 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.885357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.885641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.885917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.886622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.888266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.891397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a03c6-47cd-4bc7-a491-5f46275a0bdb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.897239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.897898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.901387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.901387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] PATH_INFO: `/v3/users/532df9b1c22b4b0c9237bb0a36044c7b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.901387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=532df9b1c22b4b0c9237bb0a36044c7b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.903145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.906879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1028/2083] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:22.911203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.913390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:22.929371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5df8-d359-43df-8092-9756f94971fc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:22.936361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:22.937149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.937496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.937844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.938637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:22.940721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:22.968784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1029/2084] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:22.970000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.002048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d5e1a-7185-472a-8b8b-cc753c67e3a6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.003637 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532df9b1c22b4b0c9237bb0a36044c7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.005099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532df9b1c22b4b0c9237bb0a36044c7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.006119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532df9b1c22b4b0c9237bb0a36044c7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.011508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.012257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.012611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.013015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] PATH_INFO: `/v3/projects/c35c678ee0b84757a38702150730c2a7/users/add3e75bfbce4868bfd2fa0d7f9f8aef/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.034902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35c678ee0b84757a38702150730c2a7, user_id=add3e75bfbce4868bfd2fa0d7f9f8aef, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.038461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd207093-c974-4e27-86d7-10f655fb6850 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.045509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532df9b1c22b4b0c9237bb0a36044c7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.059995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532df9b1c22b4b0c9237bb0a36044c7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.061004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1030/2085] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:22 2022] PUT /identity/v3/projects/c35c678ee0b84757a38702150730c2a7/users/add3e75bfbce4868bfd2fa0d7f9f8aef/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:23.073111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.074803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.074919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.107137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1056/2086] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:22 2022] DELETE /identity/v3/users/532df9b1c22b4b0c9237bb0a36044c7b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:23.138613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] MFA Rules not processed for user `add3e75bfbce4868bfd2fa0d7f9f8aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:23.139820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.147231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:23.263474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1031/2087] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:23.273990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.275511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.358677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23e12-f864-4048-9470-be1399ad68ad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.367471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.367658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.368149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.368428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.368694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.369137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.369363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.371254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.393437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332128-d89a-4088-a5a0-c1108d254680 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.401469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.402434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.402796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.403242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] PATH_INFO: `/v3/projects/e48cfdacb9c44bdd9fe7df95ee93f65b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.405060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48cfdacb9c44bdd9fe7df95ee93f65b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.407329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.410388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1057/2088] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:23.414195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.469149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48cfdacb9c44bdd9fe7df95ee93f65b', 'request_id': 'req-488e23df-aaec-4a25-9504-8ee57cacad98'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.475911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48cfdacb9c44bdd9fe7df95ee93f65b', 'request_id': 'req-488e23df-aaec-4a25-9504-8ee57cacad98'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.484954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48cfdacb9c44bdd9fe7df95ee93f65b', 'request_id': 'req-488e23df-aaec-4a25-9504-8ee57cacad98'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.487668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835b05-efad-4441-a98a-21f45079c3dc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.502295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:23.503824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] The token cache is being invalidate because project e48cfdacb9c44bdd9fe7df95ee93f65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:23.504218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.504964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-488e23df-aaec-4a25-9504-8ee57cacad98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:23.506781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1032/2089] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:23 2022] DELETE /identity/v3/projects/e48cfdacb9c44bdd9fe7df95ee93f65b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:23.511859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cf24-7ab1-4000-97d4-6d33626ebbb0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.512177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cf24-7ab1-4000-97d4-6d33626ebbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.512456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cf24-7ab1-4000-97d4-6d33626ebbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.537195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-04a4cf24-7ab1-4000-97d4-6d33626ebbb0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:23.538475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04a4cf24-7ab1-4000-97d4-6d33626ebbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.559148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.559692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.560059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.560365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.561081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.563714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.599833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1033/2090] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:23.609697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.609982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.610248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.610967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1058/2091] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:23.614883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.616513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.629182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5edce-25ce-4822-8935-58ffb5f9db33 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.630599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:23.631891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.634089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.634705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.634970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.635240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.635928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.637738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.641273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1034/2092] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:23.645335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.651236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1059/2093] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:23.654453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82187c2b-ac4b-428c-ae4c-ab8faebe98a2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.655853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.659210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.659823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.660132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.660405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.661097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:23.661395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.663005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.666322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac743a-81e5-4492-9acc-a391eba615e4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.672492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.673524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.673871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.674156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] PATH_INFO: `/v3/projects/f39943a34d994248a952f7a3f73ef1a9/users/46c0c7e688384756a7a9de547d452210/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.675374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1035/2094] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:26:23.679781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.690170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff28b-e013-4765-b854-a3ef6aff9786 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.692673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39943a34d994248a952f7a3f73ef1a9, user_id=46c0c7e688384756a7a9de547d452210, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.694714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.695314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.695448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.695588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.695857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.696551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.698524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.710463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:23.711516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1060/2095] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:23 2022] PUT /identity/v3/projects/f39943a34d994248a952f7a3f73ef1a9/users/46c0c7e688384756a7a9de547d452210/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:23.712386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.728484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1036/2096] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:23.732505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.777083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c606-14db-4801-9851-29aadfd16ed3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.796823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265a15-7406-4113-ae87-71ea3dc39878 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.801849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.802474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.802749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.803018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.803690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.805637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.828712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1037/2097] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:23.832574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.841943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.842578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.842857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.843138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.843833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.844062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0988af1-aced-43f7-a212-f9a43fa9a7f0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.845469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.848802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.849405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.849682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.849950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.850637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.852222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.858425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1061/2098] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:23.860491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.865499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1038/2099] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:23.866294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.870961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987169bf-315c-49b4-bb1c-f09435bc8b05 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.876575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.877241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.877522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.877755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5b90-d9bf-43e7-806d-0344aae51346 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.877850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] PATH_INFO: `/v3/projects/f39943a34d994248a952f7a3f73ef1a9/users/46c0c7e688384756a7a9de547d452210/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.879701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39943a34d994248a952f7a3f73ef1a9, user_id=46c0c7e688384756a7a9de547d452210, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.882263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.882401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546948c-782d-4637-949e-7904728dfc42 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.882859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.883136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.883408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.884141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.885805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.892836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1062/2100] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:23 2022] PUT /identity/v3/projects/f39943a34d994248a952f7a3f73ef1a9/users/46c0c7e688384756a7a9de547d452210/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:23.896139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1039/2101] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:23.902660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:23.903191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.903474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.903750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.912965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9535a0-ede6-4156-b438-a690d54fa553 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.913997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:23.940279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] MFA Rules not processed for user `46c0c7e688384756a7a9de547d452210`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:23.941914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.980340 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.981004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.981347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.981600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] PATH_INFO: `/v3/projects/3eb365800c0d4213b45804c34ebbf01f/users/e9dec61e00284c49bfd60edb7076c3a2/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.000158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb365800c0d4213b45804c34ebbf01f, user_id=e9dec61e00284c49bfd60edb7076c3a2, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:24.003010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1a72-4bba-4f06-9574-1fbd02797472 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:24.038397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1040/2102] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:23 2022] PUT /identity/v3/projects/3eb365800c0d4213b45804c34ebbf01f/users/e9dec61e00284c49bfd60edb7076c3a2/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:24.039062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1063/2103] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:24.040663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:24.040663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:24.040836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.048395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:24.049965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:24.071030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] MFA Rules not processed for user `e9dec61e00284c49bfd60edb7076c3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:24.072271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:24.104795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:24.110140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b767-5419-4b95-a337-0d73928e9b77 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:24.141729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1041/2104] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:24.152864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:24.154207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:24.162813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:24.163465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:24.163745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:24.164055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.165893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:24.242281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:24.242281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:24.242537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e522c0b-b441-427b-a1f7-1cb7c2fc2818 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:24.245922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:24.251622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1064/2105] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:24.257223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:24.257957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:24.258238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:24.258508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.266847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:24.268569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202fcbd-a187-4d33-8f0d-04cdaa3ae8de service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:24.281536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1042/2106] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:24.714391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:24.775575 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf369-39f0-4f19-b707-1e1aae03f50b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:24.779999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:24.780650 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:24.780928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:24.781198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] PATH_INFO: `/v3/users/e9dec61e00284c49bfd60edb7076c3a2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.782323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9dec61e00284c49bfd60edb7076c3a2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:24.784271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:24.830187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9dec61e00284c49bfd60edb7076c3a2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:24.830843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9dec61e00284c49bfd60edb7076c3a2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:24.831426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9dec61e00284c49bfd60edb7076c3a2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:24.842933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9dec61e00284c49bfd60edb7076c3a2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:24.850059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9dec61e00284c49bfd60edb7076c3a2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:24.863787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1065/2107] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:24 2022] DELETE /identity/v3/users/e9dec61e00284c49bfd60edb7076c3a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:24.868335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:24.868619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:24.868890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:24.887485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:24.888749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:24.946515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1043/2108] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:24.956492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:24.957956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.020946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2914-75f4-4ae9-a10e-761bb78efb42 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.025171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.025792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.026085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.026428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.040448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.042019 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f8d2f-a5df-4f2a-9b0b-80aff3855936 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.055869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1066/2109] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:25.340923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.342343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.355996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6d004-10da-457d-93ae-ede7d51f9ac2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.360963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.361574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.361874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.362108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] PATH_INFO: `/v3/projects/3eb365800c0d4213b45804c34ebbf01f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.363298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb365800c0d4213b45804c34ebbf01f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.365238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.420089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb365800c0d4213b45804c34ebbf01f', 'request_id': 'req-4d4ed417-a1cd-4f67-81b8-dee704899e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:25.421509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:25.427184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb365800c0d4213b45804c34ebbf01f', 'request_id': 'req-4d4ed417-a1cd-4f67-81b8-dee704899e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:25.434316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb365800c0d4213b45804c34ebbf01f', 'request_id': 'req-4d4ed417-a1cd-4f67-81b8-dee704899e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:25.450867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] The token cache is being invalidate because project 3eb365800c0d4213b45804c34ebbf01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:25.451159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:25.451784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d4ed417-a1cd-4f67-81b8-dee704899e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:25.454027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1044/2110] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:25 2022] DELETE /identity/v3/projects/3eb365800c0d4213b45804c34ebbf01f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:25.458896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.459179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.459457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.479077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:25.480774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.493715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:25.553377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1067/2111] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:25.555037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.556923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.557501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.583718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:25.585003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.595604 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1045/2112] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:25.600011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.610356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615d32d-9ebc-492e-aefc-3274565393f7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.617409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.621472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.621472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.621472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.621472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:25.621817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.623921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.653225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1068/2113] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:26:25.653827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.665470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918234-c0c6-465e-99b1-cac8b19c924a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.670816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.680171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.734956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1046/2114] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:25 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:25.735465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.748180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76022a26-d83e-408f-bcf2-a2c46f16081d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.847985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.848601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.848879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.849203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.849914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.851835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.875675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.877219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1069/2115] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:25 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:25.880411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.881070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.890779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c642-ca74-4999-9a0f-4e981f390ddd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.895526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.896194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.896476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.896750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.897437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.899070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.909819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1070/2116] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:25.913935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.924998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c7237-064d-4cb7-a1a2-5467c3b6cfd7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.929985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.930591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.930871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.931144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.931825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.933764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:25.945775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a046f-a78a-4984-94bc-4614b36314f8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.947037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1071/2117] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:25.949786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.950560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.950842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.951121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.951230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.952911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.961565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11686e-94ed-453e-9a27-7927554fd1dd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.966778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.967439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.967707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.968034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] PATH_INFO: `/v3/projects/738323a8338540e59ea9f8136825390a/users/93b8d5b88eed4992a390768824bcd5ff/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.983247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=738323a8338540e59ea9f8136825390a, user_id=93b8d5b88eed4992a390768824bcd5ff, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:25.986875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e0b1-3d76-4647-aa0a-37e829a69c3f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.000120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1072/2118] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:25 2022] PUT /identity/v3/projects/738323a8338540e59ea9f8136825390a/users/93b8d5b88eed4992a390768824bcd5ff/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:26.015307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.015583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.016677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.017148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.017309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.026169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1047/2119] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:26.053208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] MFA Rules not processed for user `93b8d5b88eed4992a390768824bcd5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:26.054463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:26.113468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:26.146718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1073/2120] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:26.151712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.165722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742aa18-02e3-4bc2-bc56-8680ed82b0bb service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.231097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.231719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.232032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.232318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.232989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.235563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.253720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:26.278730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1048/2121] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:26.283986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.295284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e0288-8710-4042-a3c8-e968f6041486 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.387672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.388345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.388630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.388907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.389594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.391561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.417108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1074/2122] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:26.422166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.434987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0033e3-3301-4fa4-8300-5c88c233787c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.442352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.442979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.443252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.443524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.444549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.446087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.460914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1049/2123] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:26.465631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.477025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab826-d573-4e9c-a0d0-0f4e98295872 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.482004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.482827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.483172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.484216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.485032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.487239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eed5d9-e32c-44e9-8c18-23cd3bfddce7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.500731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1075/2124] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:26.505187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.515876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89512a4-ede9-4fd6-8bb8-ea3f0aa8d6a6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.521459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.522130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.522442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.522695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] PATH_INFO: `/v3/projects/e59e45a0f96247cdba6ff37de0b2ad0b/users/2d53920d92ed4637a719dca11f901944/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.543717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59e45a0f96247cdba6ff37de0b2ad0b, user_id=2d53920d92ed4637a719dca11f901944, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.547141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.568798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1050/2125] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:26 2022] PUT /identity/v3/projects/e59e45a0f96247cdba6ff37de0b2ad0b/users/2d53920d92ed4637a719dca11f901944/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:26.579882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.580199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.580471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.625029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] MFA Rules not processed for user `2d53920d92ed4637a719dca11f901944`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:26.626360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:26.698166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1076/2126] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:26.703216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.721762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.766611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464880-24b4-47bd-af6e-1e831509810a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.771425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.771732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:26.772385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.772666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.773099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.773956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.776293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.779555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdc693-bd85-4d7c-9f2b-a7d0ab0ed99b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.784098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.784717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.785017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.785288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.787030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:26.807455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1051/2127] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:26.813080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.879400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.881341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.883314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa10a9-2396-4e2e-9333-4c8a0187a749 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.888465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.889125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.889411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.889687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.890884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.892822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.893413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1077/2128] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:26.916146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1052/2129] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:26 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:26.920201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.936145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a823a2-90d8-4a90-b612-2f1b4503d77a service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.942750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.943469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.943914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.944235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:26.945183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.947091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.959289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1078/2130] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:26.963613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:26.976132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e74513-95f7-49f4-a2c0-16d44bd6aaee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:26.981095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:26.981768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:26.982046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:26.982543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] PATH_INFO: `/v3/projects/a070fabef4864277ace70f3bff9e0082/users/82dc5d7182654a3b918a72b786b4d19b/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.003307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a070fabef4864277ace70f3bff9e0082, user_id=82dc5d7182654a3b918a72b786b4d19b, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:27.005998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:27.019619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1053/2131] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:26 2022] PUT /identity/v3/projects/a070fabef4864277ace70f3bff9e0082/users/82dc5d7182654a3b918a72b786b4d19b/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:27.023521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:27.034116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01ab60-aec3-457c-b9cd-25dd5dcaf5e9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:27.090055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:27.090664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:27.090945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:27.091217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.091932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:27.094183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89928a4-edb7-4c6c-ad94-40eaec65ee40 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:27.105121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1079/2132] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:27.110070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:27.116860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:27.120788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab05d2-915f-4e9c-a326-9fe725221f06 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:27.126294 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:27.126948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:27.127229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:27.127508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] PATH_INFO: `/v3/projects/a070fabef4864277ace70f3bff9e0082/users/82dc5d7182654a3b918a72b786b4d19b/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.129490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a070fabef4864277ace70f3bff9e0082, user_id=82dc5d7182654a3b918a72b786b4d19b, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:27.132151 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:27.142410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1054/2133] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:27 2022] PUT /identity/v3/projects/a070fabef4864277ace70f3bff9e0082/users/82dc5d7182654a3b918a72b786b4d19b/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:27.153967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:27.154295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:27.154609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.184192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] MFA Rules not processed for user `82dc5d7182654a3b918a72b786b4d19b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:27.185399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:27.264625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1080/2134] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:27.275601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:27.337000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961afb8-4e38-4a72-b70d-691f59c833b8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:27.341036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:27.341645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:27.341926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:27.342199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.351584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:27.353081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:27.363275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1055/2135] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:27.873832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:27.883562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d72a5-5c7d-490c-b266-61ebe8bf2ddc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:27.887150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:27.887757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:27.888064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:27.888337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:27.893523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:27.946907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:27.948383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88d41-7cd4-490d-a2bd-da2ca2f03062 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:27.957315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1081/2136] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:30.931055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:30.932635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:30.996950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b061289-518a-44f8-b666-ab0775841069 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:31.001695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:31.002326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.002609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.002884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] PATH_INFO: `/v3/users/0bb1051fa32a40a68ddc30b392a97323` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.004082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb1051fa32a40a68ddc30b392a97323)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:31.005994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:31.052403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb1051fa32a40a68ddc30b392a97323'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.053156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb1051fa32a40a68ddc30b392a97323'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.053944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb1051fa32a40a68ddc30b392a97323'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.065647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb1051fa32a40a68ddc30b392a97323'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.072345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb1051fa32a40a68ddc30b392a97323'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.085629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1056/2137] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:30 2022] DELETE /identity/v3/users/0bb1051fa32a40a68ddc30b392a97323 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:31.090204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276bb5-e422-4b71-9423-ddf5ce026815 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.090669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276bb5-e422-4b71-9423-ddf5ce026815 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.090759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276bb5-e422-4b71-9423-ddf5ce026815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.109762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-20276bb5-e422-4b71-9423-ddf5ce026815 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:31.110957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-20276bb5-e422-4b71-9423-ddf5ce026815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.169979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1082/2138] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:31.179838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:31.181312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.197149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.197487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.197811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.202015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:31.217934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] MFA Rules not processed for user `1a7627957f0c489ebf27dcab206542ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:31.219669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.266489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7431d-9ea1-459e-8bc4-0869aa4fb17d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:31.270241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:31.270878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.271158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.271432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.276515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1083/2139] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:31.281225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:31.282656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:31.292288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1057/2140] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:31.661426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:31.663060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.682728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae908fa-1e20-4e1a-9d52-db8a2eb3ada7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:31.695234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:31.695936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.696396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.697138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] PATH_INFO: `/v3/projects/adb3fb2886d4402bb0696bc84cdfe1c1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.698643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb3fb2886d4402bb0696bc84cdfe1c1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:31.700739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:31.751849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:31.753258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb3fb2886d4402bb0696bc84cdfe1c1', 'request_id': 'req-ee63208e-d7e8-45c4-bf6b-16aacc283b98'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.760060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb3fb2886d4402bb0696bc84cdfe1c1', 'request_id': 'req-ee63208e-d7e8-45c4-bf6b-16aacc283b98'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.770226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb3fb2886d4402bb0696bc84cdfe1c1', 'request_id': 'req-ee63208e-d7e8-45c4-bf6b-16aacc283b98'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.784074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] The token cache is being invalidate because project adb3fb2886d4402bb0696bc84cdfe1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:31.784074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.784074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.785647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1084/2141] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:31 2022] DELETE /identity/v3/projects/adb3fb2886d4402bb0696bc84cdfe1c1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:31.789480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:31.799385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3400d-85b1-4596-9a5d-cbaa3092a1af service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:31.857377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:31.858065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.858348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.859137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] PATH_INFO: `/v3/users/add3e75bfbce4868bfd2fa0d7f9f8aef` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.860328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add3e75bfbce4868bfd2fa0d7f9f8aef)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:31.862256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:31.908952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add3e75bfbce4868bfd2fa0d7f9f8aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.910691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add3e75bfbce4868bfd2fa0d7f9f8aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.911571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add3e75bfbce4868bfd2fa0d7f9f8aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.931499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add3e75bfbce4868bfd2fa0d7f9f8aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.940134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add3e75bfbce4868bfd2fa0d7f9f8aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:31.956103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1058/2142] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:31 2022] DELETE /identity/v3/users/add3e75bfbce4868bfd2fa0d7f9f8aef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:32.196161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:32.198499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:32.271113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63208e-d7e8-45c4-bf6b-16aacc283b98 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:32.303407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:32.305002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:32.345390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:32.346080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:32.346366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:32.346636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:32.348737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:32.359735 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:32.389227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471025b-b1fc-4f37-a69b-69db83e65c49 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:32.394802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:32.395650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:32.396122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:32.396550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] PATH_INFO: `/v3/projects/c35c678ee0b84757a38702150730c2a7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:32.398379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35c678ee0b84757a38702150730c2a7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:32.400506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:32.423656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:32.425576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:32.436430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1085/2143] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:32.466099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35c678ee0b84757a38702150730c2a7', 'request_id': 'req-75b9b2af-8410-4c65-b22c-80f6ecd380ab'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.475470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35c678ee0b84757a38702150730c2a7', 'request_id': 'req-75b9b2af-8410-4c65-b22c-80f6ecd380ab'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.485227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35c678ee0b84757a38702150730c2a7', 'request_id': 'req-75b9b2af-8410-4c65-b22c-80f6ecd380ab'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.510341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] The token cache is being invalidate because project c35c678ee0b84757a38702150730c2a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:32.510632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.511369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.512918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1059/2144] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:32 2022] DELETE /identity/v3/projects/c35c678ee0b84757a38702150730c2a7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:32.516927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:32.529672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b765c3-7df6-4bbd-8496-c396cd1c4bf5 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:32.585163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:32.607049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:32.607696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:32.608000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:32.608280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] PATH_INFO: `/v3/users/46c0c7e688384756a7a9de547d452210` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:32.609527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c0c7e688384756a7a9de547d452210)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:32.611461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:32.661579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c0c7e688384756a7a9de547d452210'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.662268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c0c7e688384756a7a9de547d452210'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.662887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c0c7e688384756a7a9de547d452210'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.676267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c0c7e688384756a7a9de547d452210'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.684390 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5968674d-0e96-4f36-a6fb-204ff53ca6c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c0c7e688384756a7a9de547d452210'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:32.700212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1086/2145] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:32 2022] DELETE /identity/v3/users/46c0c7e688384756a7a9de547d452210 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:33.110588 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.112168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:33.133923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b2af-8410-4c65-b22c-80f6ecd380ab admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.219123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.219762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.220071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.220350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] PATH_INFO: `/v3/projects/f39943a34d994248a952f7a3f73ef1a9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.221521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f39943a34d994248a952f7a3f73ef1a9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.223431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.299216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f39943a34d994248a952f7a3f73ef1a9', 'request_id': 'req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:33.305082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f39943a34d994248a952f7a3f73ef1a9', 'request_id': 'req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:33.311421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f39943a34d994248a952f7a3f73ef1a9', 'request_id': 'req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:33.316048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:33.329131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] The token cache is being invalidate because project f39943a34d994248a952f7a3f73ef1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:33.329646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:33.330686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a6eb0e9-e9ed-4e75-b103-2ddc0fca0647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:33.336260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1060/2146] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:33 2022] DELETE /identity/v3/projects/f39943a34d994248a952f7a3f73ef1a9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:33.339389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.339874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.340102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.361010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:33.362371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:33.437923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1087/2147] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:33.439638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.439938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.440289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.461418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:33.462381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:33.470634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1061/2148] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:33.475692 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.486655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a746b38-1ccb-4133-a363-74ee95938d59 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.491628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.492281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.492566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.492841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.493553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:33.493861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.495764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.510956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1088/2149] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:26:33.515828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.525980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3442-ec0e-4167-b347-289ecc87b913 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.530751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.531378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.531647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.531921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.532618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.534580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.570417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1062/2150] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:33.574704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.585306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688134-357e-4d02-8006-46e9d8ce7ead admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.637418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:33.650054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.651409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.651691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.651982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.652665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.654614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.678092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1089/2151] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:33.682351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.693415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec27b30-2707-4766-a398-c7abf283c6ef admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.698112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.698835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.699121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.699399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.700127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.701717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.711944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1063/2152] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:33.715816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.727292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a60461-5739-48f1-92cd-03627425b82d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.732149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.732764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.733167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.733397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.734928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.736729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffa28b-6b85-46fe-a7cf-da8a5f2e9bed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.751937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1090/2153] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:33.753793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.765907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bfb0a-b70a-4863-a67f-8c3be33005cf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.770358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.771027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.771330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.771606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] PATH_INFO: `/v3/projects/5084c2b61ad2427d91e0fc85392aee14/users/c1ecb46bae9e4e369416d138203abe6f/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.791844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5084c2b61ad2427d91e0fc85392aee14, user_id=c1ecb46bae9e4e369416d138203abe6f, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.795091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.811716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1064/2154] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:33 2022] PUT /identity/v3/projects/5084c2b61ad2427d91e0fc85392aee14/users/c1ecb46bae9e4e369416d138203abe6f/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:33.816886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.817250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.817548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.836267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:33.837464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:33.898290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1091/2155] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:33.909019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:33.910726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:33.969518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc15c2-612a-407c-b756-52ab4089867b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:33.973386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:33.974189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.974464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.974734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.984469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:33.986398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bb92b-1221-4ed0-9317-09f631fe9d10 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:33.996260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1065/2156] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:34.840579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:34.841567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:34.853833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034dd85-f786-4934-911b-82fd9003a2bf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:34.860823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:34.862082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:34.862677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:34.863460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] PATH_INFO: `/v3/users/93b8d5b88eed4992a390768824bcd5ff` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:34.865275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b8d5b88eed4992a390768824bcd5ff)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:34.867946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:34.938715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b8d5b88eed4992a390768824bcd5ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:34.938715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b8d5b88eed4992a390768824bcd5ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:34.938715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b8d5b88eed4992a390768824bcd5ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:34.956250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b8d5b88eed4992a390768824bcd5ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:34.964214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b8d5b88eed4992a390768824bcd5ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:34.987997 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1092/2157] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:34 2022] DELETE /identity/v3/users/93b8d5b88eed4992a390768824bcd5ff => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:34.988272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:34.988694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:34.988694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:35.013142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:35.014419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:35.081606 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1066/2158] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:35.095453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:35.097254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:35.150467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:35.167734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2498f-6751-4ae5-83b9-bdb78fb40a83 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:35.174619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:35.175372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:35.175749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:35.176164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:35.187770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:35.189563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:35.203710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1093/2159] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:35.757581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:35.759401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:35.781171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972afc-288f-4d0b-9cd9-462cec7ac06b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:35.788058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:35.789230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:35.789650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:35.790083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] PATH_INFO: `/v3/projects/738323a8338540e59ea9f8136825390a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:35.791766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738323a8338540e59ea9f8136825390a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:35.794161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:35.840474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:35.865702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738323a8338540e59ea9f8136825390a', 'request_id': 'req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:35.874492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738323a8338540e59ea9f8136825390a', 'request_id': 'req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:35.884034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738323a8338540e59ea9f8136825390a', 'request_id': 'req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:35.900913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] The token cache is being invalidate because project 738323a8338540e59ea9f8136825390a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:35.900913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:35.901697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:35.904002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1067/2160] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:35 2022] DELETE /identity/v3/projects/738323a8338540e59ea9f8136825390a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:35.909030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:35.921896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ed59b-b382-4e7b-8077-ccfc3d492ba4 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:36.025011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:36.025656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.028004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.028004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] PATH_INFO: `/v3/users/2d53920d92ed4637a719dca11f901944` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.028004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d53920d92ed4637a719dca11f901944)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:36.029608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:36.100015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d53920d92ed4637a719dca11f901944'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.101007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d53920d92ed4637a719dca11f901944'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.101798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d53920d92ed4637a719dca11f901944'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.114591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d53920d92ed4637a719dca11f901944'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.122764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-43e90ea5-8cb3-46fe-baf5-f5c3795d701f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d53920d92ed4637a719dca11f901944'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.137275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1094/2161] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:35 2022] DELETE /identity/v3/users/2d53920d92ed4637a719dca11f901944 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:36.325801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:36.328473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:36.360279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a793a6b-185f-4a7e-afbe-3d3a3d978ff7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:36.443497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:36.444412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.444791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.448405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] PATH_INFO: `/v3/users/0e434f8b9b1d48f3bd1a2071323ee217` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.449941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e434f8b9b1d48f3bd1a2071323ee217)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:36.452305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:36.516955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e434f8b9b1d48f3bd1a2071323ee217'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.517689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e434f8b9b1d48f3bd1a2071323ee217'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.518274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e434f8b9b1d48f3bd1a2071323ee217'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.540916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e434f8b9b1d48f3bd1a2071323ee217'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.549967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e434f8b9b1d48f3bd1a2071323ee217'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.569755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1068/2162] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:36 2022] DELETE /identity/v3/users/0e434f8b9b1d48f3bd1a2071323ee217 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:36.573896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.574183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.574458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.596931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:36.598147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:36.658448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:36.660454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:36.691708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1095/2163] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:36.703843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:36.705315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:36.718462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:36.739914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:36.793135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c154e-9b64-4a95-9502-b649d343d617 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:36.795451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450308-f1a8-4d08-8e81-8423c3d00f7a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:36.798003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:36.798630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.798915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.799191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.802354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:36.803178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.803629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.804079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] PATH_INFO: `/v3/projects/e59e45a0f96247cdba6ff37de0b2ad0b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.805442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59e45a0f96247cdba6ff37de0b2ad0b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:36.807755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:36.810184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:36.811701 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:36.824847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1096/2164] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:36.877360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59e45a0f96247cdba6ff37de0b2ad0b', 'request_id': 'req-267d2dcb-7759-41af-940c-ad9813aade92'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.884178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59e45a0f96247cdba6ff37de0b2ad0b', 'request_id': 'req-267d2dcb-7759-41af-940c-ad9813aade92'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.897090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59e45a0f96247cdba6ff37de0b2ad0b', 'request_id': 'req-267d2dcb-7759-41af-940c-ad9813aade92'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.919163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] The token cache is being invalidate because project e59e45a0f96247cdba6ff37de0b2ad0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:36.919767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.920863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:36.927721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:36.929158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1069/2165] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:36 2022] DELETE /identity/v3/projects/e59e45a0f96247cdba6ff37de0b2ad0b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:36.945661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573edd-e06b-4ef8-97c0-f5e999e58521 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:37.024888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:37.025552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:37.025834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:37.026109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] PATH_INFO: `/v3/users/82dc5d7182654a3b918a72b786b4d19b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:37.027571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] RBAC: Authorizing `identity:delete_user(user_id=82dc5d7182654a3b918a72b786b4d19b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:37.031799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:37.097345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82dc5d7182654a3b918a72b786b4d19b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:37.098122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82dc5d7182654a3b918a72b786b4d19b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:37.098889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82dc5d7182654a3b918a72b786b4d19b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:37.121428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82dc5d7182654a3b918a72b786b4d19b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:37.131123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82dc5d7182654a3b918a72b786b4d19b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:37.146352 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1097/2166] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:36 2022] DELETE /identity/v3/users/82dc5d7182654a3b918a72b786b4d19b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:37.819245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:37.820683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:37.834678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d2dcb-7759-41af-940c-ad9813aade92 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:37.904869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:37.907557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:37.907869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:37.908222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] PATH_INFO: `/v3/projects/e87994ec0444411f9866120d041cc50a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:37.909678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87994ec0444411f9866120d041cc50a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:37.911653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:37.949083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:37.950592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:37.970402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c0e47-7136-4137-8b51-8c699110423e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:37.998180 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87994ec0444411f9866120d041cc50a', 'request_id': 'req-fe413876-bc43-4b26-8af0-1923a0add806'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.003110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:38.012976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87994ec0444411f9866120d041cc50a', 'request_id': 'req-fe413876-bc43-4b26-8af0-1923a0add806'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.020965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87994ec0444411f9866120d041cc50a', 'request_id': 'req-fe413876-bc43-4b26-8af0-1923a0add806'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.030720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:38.038582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] The token cache is being invalidate because project e87994ec0444411f9866120d041cc50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:38.038851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.039432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fe413876-bc43-4b26-8af0-1923a0add806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.041609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1070/2167] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:37 2022] DELETE /identity/v3/projects/e87994ec0444411f9866120d041cc50a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:38.043362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.044048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.044334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.044610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] PATH_INFO: `/v3/projects/a070fabef4864277ace70f3bff9e0082` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.046861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a070fabef4864277ace70f3bff9e0082)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.049446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bec1c-c32e-4b6a-9d66-13d26358858b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.049576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.049855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bec1c-c32e-4b6a-9d66-13d26358858b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.050331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bec1c-c32e-4b6a-9d66-13d26358858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.073885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-451bec1c-c32e-4b6a-9d66-13d26358858b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:38.075210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-451bec1c-c32e-4b6a-9d66-13d26358858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:38.084358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1071/2168] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:38.110214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a070fabef4864277ace70f3bff9e0082', 'request_id': 'req-e663771b-e376-4a43-bb12-f7d46f045036'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.117635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a070fabef4864277ace70f3bff9e0082', 'request_id': 'req-e663771b-e376-4a43-bb12-f7d46f045036'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.130021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a070fabef4864277ace70f3bff9e0082', 'request_id': 'req-e663771b-e376-4a43-bb12-f7d46f045036'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.145092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] The token cache is being invalidate because project a070fabef4864277ace70f3bff9e0082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:38.145480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.146258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e663771b-e376-4a43-bb12-f7d46f045036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:38.148719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1098/2169] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:37 2022] DELETE /identity/v3/projects/a070fabef4864277ace70f3bff9e0082 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:38.154992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247221-4a66-4af2-919a-a7dbef7ca1e2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.155415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247221-4a66-4af2-919a-a7dbef7ca1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.155621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247221-4a66-4af2-919a-a7dbef7ca1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.161276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.161568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.161857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.186242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fc247221-4a66-4af2-919a-a7dbef7ca1e2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:38.190649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fc247221-4a66-4af2-919a-a7dbef7ca1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:38.196881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:38.198963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:38.277084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1099/2170] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:38.281480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.291180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6d474-550f-47d9-b26d-6f4518956fb9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.297515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.298604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.299132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.299567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.300457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:38.300934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.302999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.317144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1072/2171] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:38.318652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.318652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.318861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.323819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1100/2172] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:26:38.326049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.337060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3a61-b213-4877-abd8-6478affeb8a5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.343344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:38.344672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:38.357372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1073/2173] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:38.359849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.361786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.364504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.364956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.366533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.367929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.370707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.372372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd337-e5b8-411a-b27f-68635df8c214 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.377584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.378419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.378704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.378971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.379908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:38.380299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.381973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.404051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1074/2174] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:38.405004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.410060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1101/2175] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:38.414388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.417806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9423277-0a26-48a4-b682-378ba993320a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.471858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:38.517906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.518525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.518808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.519081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.519758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.521719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.531210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469667e-0636-4071-aa87-00da567ada4a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.537042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.538180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.538467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.538730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.540282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.543355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.597410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1075/2176] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:38.603209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1102/2177] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:38.607307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.608788 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.634828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41241f-0006-4cbc-bd24-5be5ab0ede32 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.730950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:38.740338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfea93-522f-4a8a-abc4-0c8fa6f034d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.758577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.759392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.759901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.760376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.761949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.764381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.777352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1076/2178] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:38.781858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.794527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0eef-caef-4536-9cc9-bfb8e2d106d9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.805350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.805989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.806544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.806544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.807561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.808904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.809677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.809920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.810300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.810765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.811987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.813816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.830946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1077/2179] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:38.834973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.850308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a323-83e9-4236-b730-0d6176aece76 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.852862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1103/2180] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:38.859344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.868720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.869338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.869625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.871228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] PATH_INFO: `/v3/projects/25fefc765ecb42728a941ddad185e082/users/f20047857e8c4e36a031813729c09a2b/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.877935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c8cf4-d803-42e5-b651-e97fb57081f7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.891381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:38.892047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.892334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.892611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.894786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.898309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.908473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fefc765ecb42728a941ddad185e082, user_id=f20047857e8c4e36a031813729c09a2b, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:38.914008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefadf7-d100-42e1-a001-d56e49f49a24 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:38.919432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1104/2181] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:38.927017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:38.942840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1078/2182] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:38 2022] PUT /identity/v3/projects/25fefc765ecb42728a941ddad185e082/users/f20047857e8c4e36a031813729c09a2b/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:38.944890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeb8c0-c09d-40c0-8949-a6c0474de768 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:38.948334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:38.948629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:38.948950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:38.987307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:38.989228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.052334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.053739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.054300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.054789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.056612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.060275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.067495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1079/2183] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:39.068578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.070085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.079791 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1105/2184] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:39.081335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.096722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02cf0f-35aa-4ed5-a0fd-69e5cb2bed49 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.103907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.105013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.105502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.105977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] PATH_INFO: `/v3/projects/b39c69a5c6a14cd6b88c0de91bb65f83/users/f470fe1baa4e460ab3cfe3f0c5c4efbc/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.128760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b39c69a5c6a14cd6b88c0de91bb65f83, user_id=f470fe1baa4e460ab3cfe3f0c5c4efbc, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.132465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd766e0d-a0fd-4cf1-b228-065b2977d15d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.137809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d5a8-19ff-4b5a-9f66-f4ac60281d70 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.142379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.143040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.143289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.143564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.154804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1106/2185] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:39 2022] PUT /identity/v3/projects/b39c69a5c6a14cd6b88c0de91bb65f83/users/f470fe1baa4e460ab3cfe3f0c5c4efbc/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:39.155020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.157215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.166540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.166826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.167100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.169241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1080/2186] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:39.176283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:39.202211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] MFA Rules not processed for user `f470fe1baa4e460ab3cfe3f0c5c4efbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:39.203504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.307991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1107/2187] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:39.312590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.326058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ffbb-f00c-4776-b770-c5fbce974c73 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.446140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.446873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.447389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.448699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.449727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.452552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.471613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:39.526203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1081/2188] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:39 2022] POST /identity/v3/projects => generated 389 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:26:39.531197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.549403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5b924-9e72-4e4c-9a74-d05a5638065c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.667286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.668740 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.669098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.670201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.671849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.694486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.730786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1108/2189] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:39 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:39.735848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.747608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c350e9-ad0b-4eba-9a68-a8304f401ef3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.754338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.755139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.755569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.755935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.756845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.758770 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.772005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1082/2190] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:39.776492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.790731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db449f6-9c25-46b0-87d2-aae7661d27c1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.817022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.818741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.819344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.820715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] PATH_INFO: `/v3/projects/e28fb18fb5d2473babd8776e72a462d7/users/eaab8e838be648908164dfb785ccc80a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.849037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28fb18fb5d2473babd8776e72a462d7, user_id=eaab8e838be648908164dfb785ccc80a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.854168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.875036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1109/2191] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:39 2022] PUT /identity/v3/projects/e28fb18fb5d2473babd8776e72a462d7/users/eaab8e838be648908164dfb785ccc80a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:39.881009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.901645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94cf5-7f43-4621-af5d-cb9bad2e23d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:40.019121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:40.021234 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.022886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.023188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.025162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:40.029852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82799e-f18b-42f1-94cf-1e31b59a67eb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:40.058590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1083/2192] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:26:40.060663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:40.099801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b28a7-68bf-4d75-95f5-458d5764c553 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:40.115064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:40.116935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.117445 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.119308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] PATH_INFO: `/v3/projects/e28fb18fb5d2473babd8776e72a462d7/users/eaab8e838be648908164dfb785ccc80a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.124718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28fb18fb5d2473babd8776e72a462d7, user_id=eaab8e838be648908164dfb785ccc80a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:40.129446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933334da-38b6-4d30-8382-cd1848d969dc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:40.162397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1110/2193] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:40 2022] PUT /identity/v3/projects/e28fb18fb5d2473babd8776e72a462d7/users/eaab8e838be648908164dfb785ccc80a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:40.175211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.175609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.175765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.265991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] MFA Rules not processed for user `eaab8e838be648908164dfb785ccc80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:40.267464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:40.337472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.337840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.338639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.414698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] MFA Rules not processed for user `f20047857e8c4e36a031813729c09a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:40.426917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:40.450312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1084/2194] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:40.464201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:40.465867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:40.561232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:40.625644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1111/2195] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:40.637132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:40.637904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8a1f9-36a3-4e78-b3ff-08c35d34a74b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:40.638698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:40.646959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:40.648615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.648894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.649167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.665243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e0da4-14c1-4354-9070-88f373bda9b2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:40.670004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:40.670633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.671691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.671949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.679298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:40.681336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f5b47-42a9-40ef-a02c-17e5c4525531 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:40.689065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:40.690923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:40.692556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:40.698670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1085/2196] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:40.713043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1112/2197] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:40.867140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:40.877094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:40.879645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:40.915548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:40.927367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] MFA Rules not processed for user `c1ecb46bae9e4e369416d138203abe6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:40.928779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:41.023757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8f3d-9096-49c1-b902-be827995e389 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.029146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.030156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.030450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.030729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] PATH_INFO: `/v3/users/f470fe1baa4e460ab3cfe3f0c5c4efbc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.033133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f470fe1baa4e460ab3cfe3f0c5c4efbc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.035249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.037756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1086/2198] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:41.038677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.040350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:41.057344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc7b0-02d1-43f7-baec-ca46bf38b60a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.075935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.077549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.077836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.078107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.078795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.081166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.113748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f470fe1baa4e460ab3cfe3f0c5c4efbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:41.115339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f470fe1baa4e460ab3cfe3f0c5c4efbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:41.116577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f470fe1baa4e460ab3cfe3f0c5c4efbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:41.144017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1087/2199] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:41 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:41.146412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.148344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:41.154115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f470fe1baa4e460ab3cfe3f0c5c4efbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:41.171688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-554a443b-a41c-43ee-bf5e-a6a473b85c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f470fe1baa4e460ab3cfe3f0c5c4efbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:41.198542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1113/2200] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:40 2022] DELETE /identity/v3/users/f470fe1baa4e460ab3cfe3f0c5c4efbc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:26:41.201431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.201787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.202990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.235502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:41.240744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:41.262196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421d34c-a5d2-4666-a39f-ea0ba0439c9c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.315436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:41.353419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.354112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.354654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.354997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.355753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.357225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1114/2201] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:41.358044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.367559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.369457 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:41.393510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1088/2202] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:41 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:41.397974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.408813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:41.414825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc86d-a569-4195-bf81-b842b13d7abb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.428038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.429016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.429498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.429997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.430885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.432752 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.455337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1089/2203] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:41.456988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.465466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07245bc3-dd11-4f92-9c2c-50aa81cf709f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.469767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.471265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.471556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.471831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.475701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1a1d9-1da4-4fbb-9bb6-8d31f50906b8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.484812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.487183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.489125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.490157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.490662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.491143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] PATH_INFO: `/v3/projects/dbc945bd369441218e963648aab071d7/users/f8d0c3bb819441caafac005f5b7463de/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.498750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1115/2204] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:26:41.542640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc945bd369441218e963648aab071d7, user_id=f8d0c3bb819441caafac005f5b7463de, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.545333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.572748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1090/2205] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:41 2022] PUT /identity/v3/projects/dbc945bd369441218e963648aab071d7/users/f8d0c3bb819441caafac005f5b7463de/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:41.576584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.594589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c8fb-a505-499f-a958-2f2cd5f410ab service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.711843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.712763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.712940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.713399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.714396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.716165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.729080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1116/2206] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:41.734038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:41.745066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5509078-4dd4-46c0-9d0d-2d128e1a7051 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:41.752554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:41.753548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:41.753826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:41.754098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] PATH_INFO: `/v3/projects/dbc945bd369441218e963648aab071d7/users/f8d0c3bb819441caafac005f5b7463de/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:41.756755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc945bd369441218e963648aab071d7, user_id=f8d0c3bb819441caafac005f5b7463de, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:41.758895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b5cc-9195-462c-a649-e98b36808e4e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:41.775545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1091/2207] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:41 2022] PUT /identity/v3/projects/dbc945bd369441218e963648aab071d7/users/f8d0c3bb819441caafac005f5b7463de/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:42.256277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:42.261123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:42.280418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af483fc3-744c-48a0-806f-410662faba82 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:42.366286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:42.367135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.367476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.367936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] PATH_INFO: `/v3/projects/b39c69a5c6a14cd6b88c0de91bb65f83` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.370303 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b39c69a5c6a14cd6b88c0de91bb65f83)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:42.376836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:42.378974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.378974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.378974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.407477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] MFA Rules not processed for user `f20047857e8c4e36a031813729c09a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:42.408891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:42.457825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:42.511053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b39c69a5c6a14cd6b88c0de91bb65f83', 'request_id': 'req-a7f8865b-27c7-4314-a48c-47e4a01a88a0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.522392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b39c69a5c6a14cd6b88c0de91bb65f83', 'request_id': 'req-a7f8865b-27c7-4314-a48c-47e4a01a88a0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.533388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b39c69a5c6a14cd6b88c0de91bb65f83', 'request_id': 'req-a7f8865b-27c7-4314-a48c-47e4a01a88a0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.546411 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1092/2208] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:42.549551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] The token cache is being invalidate because project b39c69a5c6a14cd6b88c0de91bb65f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:42.549551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.550292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.553044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1117/2209] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:42 2022] DELETE /identity/v3/projects/b39c69a5c6a14cd6b88c0de91bb65f83 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:42.556802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:42.571913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f603b6-0579-47b9-80c9-b6530d4c9629 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:42.674081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:42.674740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.675030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.675292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] PATH_INFO: `/v3/users/eaab8e838be648908164dfb785ccc80a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.676529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaab8e838be648908164dfb785ccc80a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:42.678466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:42.741195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaab8e838be648908164dfb785ccc80a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.743551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaab8e838be648908164dfb785ccc80a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.745101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaab8e838be648908164dfb785ccc80a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.773457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:42.775528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaab8e838be648908164dfb785ccc80a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.787361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26cafad6-2eaa-4c59-8e28-9a83eb7512f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaab8e838be648908164dfb785ccc80a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:42.808755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1093/2210] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:42 2022] DELETE /identity/v3/users/eaab8e838be648908164dfb785ccc80a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:43.229598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:43.238525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:43.266905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8865b-27c7-4314-a48c-47e4a01a88a0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:43.384334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:43.385776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:43.386246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:43.386670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] PATH_INFO: `/v3/projects/e28fb18fb5d2473babd8776e72a462d7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:43.388971 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28fb18fb5d2473babd8776e72a462d7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:43.393416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:43.499585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28fb18fb5d2473babd8776e72a462d7', 'request_id': 'req-1bdaa865-586f-4d18-93f3-9c64bec94e42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:43.515626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28fb18fb5d2473babd8776e72a462d7', 'request_id': 'req-1bdaa865-586f-4d18-93f3-9c64bec94e42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:43.543988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28fb18fb5d2473babd8776e72a462d7', 'request_id': 'req-1bdaa865-586f-4d18-93f3-9c64bec94e42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:43.566671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] The token cache is being invalidate because project e28fb18fb5d2473babd8776e72a462d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:43.566964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:43.572598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1bdaa865-586f-4d18-93f3-9c64bec94e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:43.574492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1118/2211] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:43 2022] DELETE /identity/v3/projects/e28fb18fb5d2473babd8776e72a462d7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:43.583474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:43.583941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:43.584189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:43.615247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:43.616644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:43.697159 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1094/2212] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:43.701271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:43.701616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:43.701944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:43.757075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:43.758877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:43.785823 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1119/2213] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:43.789930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:43.803044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149eec98-9929-4d15-b5e4-d0a353c4962b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:43.817208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:43.817847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:43.818124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:43.818393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:43.819093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:43.819397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:43.821028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:43.848834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:43.854600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1095/2214] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:43.859214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6800a-4c31-4b46-8844-af44d51f4118 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:43.870768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:43.871530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:43.871986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:43.872236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:43.872975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:43.876768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:43.947376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:43.948248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1120/2215] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:43 2022] POST /identity/v3/projects => generated 395 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:43.961967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a23c3-d790-4dbc-9a1f-160b2d1a19de admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:44.069734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:44.070462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.070931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.071338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.072284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:44.074637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:44.128253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:44.130946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1096/2216] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:43 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:44.144844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7e2f-b0d9-41b1-8811-6fd7bb54188d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:44.153837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:44.154844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.156123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.156796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.157784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:44.160609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:44.186040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1121/2217] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:44.191339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:44.229687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d07709-59c9-42cf-b50e-83402051d09c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:44.284315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:44.284958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.285275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.285653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.286887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:44.300317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0a9bf-fa27-4548-85a9-11c638515dd7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:44.328433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1097/2218] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:44.332416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:44.338849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:44.345446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a77a7-9160-4448-935b-630ea6ec4898 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:44.356394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:44.357345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.357934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.358443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] PATH_INFO: `/v3/projects/427ac64bd0114b8987ee810ee4fa90df/users/9d63f5bacb494e38ae4290177dde90a4/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.397151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427ac64bd0114b8987ee810ee4fa90df, user_id=9d63f5bacb494e38ae4290177dde90a4, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:44.402838 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:44.427546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1122/2219] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:44 2022] PUT /identity/v3/projects/427ac64bd0114b8987ee810ee4fa90df/users/9d63f5bacb494e38ae4290177dde90a4/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:44.439428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddd87-ab9f-449e-8de6-aecb3b693d4a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.439939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddd87-ab9f-449e-8de6-aecb3b693d4a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.440558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddd87-ab9f-449e-8de6-aecb3b693d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.533541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f3eddd87-ab9f-449e-8de6-aecb3b693d4a None None] MFA Rules not processed for user `9d63f5bacb494e38ae4290177dde90a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:44.534974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f3eddd87-ab9f-449e-8de6-aecb3b693d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:44.615261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:44.663369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1098/2220] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:44.677937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:44.679738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:44.798988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23a3d7-5133-4101-b533-756d18ae1487 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:44.803293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:44.804151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:44.804587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:44.805040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:44.821383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:44.825075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:44.858857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1123/2221] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:44 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:49.202407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5117d6-1846-49c6-836e-fd018bebae20 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:49.202992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5117d6-1846-49c6-836e-fd018bebae20 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:49.202992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5117d6-1846-49c6-836e-fd018bebae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:49.233970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ba5117d6-1846-49c6-836e-fd018bebae20 None None] MFA Rules not processed for user `f8d0c3bb819441caafac005f5b7463de`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:49.235186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba5117d6-1846-49c6-836e-fd018bebae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:49.318760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1099/2222] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:49.333123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:49.334644 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:49.410367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a754e-5d24-477e-9066-ceb9a2f5e294 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:49.415800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:49.416444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:49.416726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:49.417002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:49.418733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:49.476043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:49.477624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:49.487829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1124/2223] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:50.638169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff4ce-9e47-4596-b503-db88f70d7ba1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:50.639176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff4ce-9e47-4596-b503-db88f70d7ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:50.640513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff4ce-9e47-4596-b503-db88f70d7ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:50.664632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f66ff4ce-9e47-4596-b503-db88f70d7ba1 None None] MFA Rules not processed for user `c1ecb46bae9e4e369416d138203abe6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:50.666089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f66ff4ce-9e47-4596-b503-db88f70d7ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:50.678688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1100/2224] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:54.335369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:54.354417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:54.400871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88dc75-76fc-4b89-9a56-f76c7ecaa933 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:54.406973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:54.407720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:54.408034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:54.408795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] PATH_INFO: `/v3/users/9d63f5bacb494e38ae4290177dde90a4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:54.409694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d63f5bacb494e38ae4290177dde90a4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:54.411632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:54.474029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d63f5bacb494e38ae4290177dde90a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:54.474782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d63f5bacb494e38ae4290177dde90a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:54.475470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d63f5bacb494e38ae4290177dde90a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:54.492014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d63f5bacb494e38ae4290177dde90a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:54.501214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d63f5bacb494e38ae4290177dde90a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:54.516017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1125/2225] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:26:54 2022] DELETE /identity/v3/users/9d63f5bacb494e38ae4290177dde90a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:54.520474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:54.521004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:54.521099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:54.541022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:54.542264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:54.613959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1101/2226] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:54.623828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:54.625764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:54.688599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa94c-e9b0-4809-b117-ecbca843aa92 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:54.692386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:54.693004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:54.693287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:54.693567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:54.703606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:54.705196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5fae0-45f6-4967-bbfd-6f3a1088eff7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:54.715357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1126/2227] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:54.999838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.001638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:55.015086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974dd9a-bfd0-474a-b75c-ade8cc5b137a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.019684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.020467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.020751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.021026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] PATH_INFO: `/v3/projects/427ac64bd0114b8987ee810ee4fa90df` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.022179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=427ac64bd0114b8987ee810ee4fa90df)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.024113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.075693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427ac64bd0114b8987ee810ee4fa90df', 'request_id': 'req-d57d0c88-d36e-4345-9aea-4dcb30d68738'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:55.077562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:55.082865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427ac64bd0114b8987ee810ee4fa90df', 'request_id': 'req-d57d0c88-d36e-4345-9aea-4dcb30d68738'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:55.090054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427ac64bd0114b8987ee810ee4fa90df', 'request_id': 'req-d57d0c88-d36e-4345-9aea-4dcb30d68738'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:55.103208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] The token cache is being invalidate because project 427ac64bd0114b8987ee810ee4fa90df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:26:55.103695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:55.104382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d57d0c88-d36e-4345-9aea-4dcb30d68738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:26:55.106000 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1102/2228] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:26:54 2022] DELETE /identity/v3/projects/427ac64bd0114b8987ee810ee4fa90df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:55.111765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.112082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.112375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.133172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:55.134465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:55.169092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:55.197539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1127/2229] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:26:55.205625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.205942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.206221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.227084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:55.228324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:55.236696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1103/2230] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:55.241146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.252883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27197-d1d4-43aa-83ae-cd184fb5294a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.257821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.258449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.258849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.259129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.259845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:26:55.260199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.262726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.275311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1128/2231] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:26:55.279773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.289912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1887190-e43a-481c-81c3-5338a7ce7007 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.294981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.296179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.296461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.296745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.297415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.299584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.327947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1104/2232] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:26:55 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:55.332190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.342151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3d41a-3746-41ea-9616-7c1a08bca164 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.399092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.399688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.399981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.400260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.401350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.403026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.424189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1129/2233] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:26:55 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:26:55.428715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.439619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e9f4-c386-458f-988c-21bc08545724 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.444208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.444839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.445113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.445378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.446062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.447623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.457508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1105/2234] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:55.461623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.472427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb159400-7e2e-4a3a-99d4-640182d0e337 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.477200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.477812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.478096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.478367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.479065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.480737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29826b-ff16-4858-b8a8-7c87b0b32a75 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.493325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1130/2235] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:26:55.497534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.507893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2b34b-4419-4a67-9a0b-f1fedec212c5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.513231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.513919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.514244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.514530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] PATH_INFO: `/v3/projects/82b077dd80c54b08ba3ad79b8cafd3f7/users/fd829e78e710440687678701fea3cf2c/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.530622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b077dd80c54b08ba3ad79b8cafd3f7, user_id=fd829e78e710440687678701fea3cf2c, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.533328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.546790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1106/2236] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:26:55 2022] PUT /identity/v3/projects/82b077dd80c54b08ba3ad79b8cafd3f7/users/fd829e78e710440687678701fea3cf2c/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:26:55.552298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003d383-d749-45b6-9480-27ff3eec2e45 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.552587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003d383-d749-45b6-9480-27ff3eec2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.552863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003d383-d749-45b6-9480-27ff3eec2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.571324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2003d383-d749-45b6-9480-27ff3eec2e45 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:55.572543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2003d383-d749-45b6-9480-27ff3eec2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:55.635458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1131/2237] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:55.641977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:55.643702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:55.703933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8347150-f3ff-4fb1-82c7-4d1f4abb5316 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:55.707568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:55.708239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:55.708521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:55.708791 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:55.718961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:55.720412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:55.729780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1107/2238] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:58.799006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216f10-52c2-4272-b6e4-0aac6fd7f287 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:58.799853 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216f10-52c2-4272-b6e4-0aac6fd7f287 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:58.800016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216f10-52c2-4272-b6e4-0aac6fd7f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:58.827345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d6216f10-52c2-4272-b6e4-0aac6fd7f287 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:58.827345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d6216f10-52c2-4272-b6e4-0aac6fd7f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:58.899386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:58.919305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1132/2239] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:58.986077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:58.987645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:59.094597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049962-cf82-4586-a2ef-25bddab93c07 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:59.101146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:59.101977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:59.102258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:59.102755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:59.117639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:59.119663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:59.130531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1108/2240] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:26:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:27:01.242490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:01.243082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:01.243217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:01.312890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] MFA Rules not processed for user `fd829e78e710440687678701fea3cf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:01.314466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:01.472267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1133/2241] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:01.478974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:01.509017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916078-e1f9-4d93-9533-61fe21400db4 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:01.521601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:01.522649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:01.522952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:01.523244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:01.524122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:01.526025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:01.535619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:01.577939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1109/2242] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:27:01 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:27:01.582475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:01.599552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91593c-bbd6-4c5c-8d48-1565d177b64b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:01.672393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:01.673334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:01.673617 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:01.673893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:01.674563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:01.676702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:01.705948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1134/2243] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:27:01 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:27:01.706919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:01.720664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992945c0-e5e0-4b92-8a18-757b4cab777e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:01.736830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:01.741897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:01.743363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:01.743661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:01.744424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:01.752477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:01.780344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1110/2244] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:27:01.784889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:01.796919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e079bd9-ee6a-473c-9e73-40c60b9955d5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:01.816397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:01.817369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:01.817811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:01.817931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] PATH_INFO: `/v3/projects/bb80a63667074f8f94417ad29b54bda3/users/d20f5320adee461e9e90bc4465186913/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:01.851030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb80a63667074f8f94417ad29b54bda3, user_id=d20f5320adee461e9e90bc4465186913, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:01.861459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:01.905905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1135/2245] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:27:01 2022] PUT /identity/v3/projects/bb80a63667074f8f94417ad29b54bda3/users/d20f5320adee461e9e90bc4465186913/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:27:01.909988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:01.943213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e3b8-7411-43e9-aa88-4f672dd6b62b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:02.102568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:02.103347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:02.103699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:02.104091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:02.104862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:02.106637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4a8b-5150-43f2-80eb-1d93a388611b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:02.143093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1111/2246] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:27:02.147436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:02.176211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc00a9-dbdd-4220-952a-364d04561298 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:02.195736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:02.196667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:02.196965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:02.197459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] PATH_INFO: `/v3/projects/bb80a63667074f8f94417ad29b54bda3/users/d20f5320adee461e9e90bc4465186913/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:02.200406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb80a63667074f8f94417ad29b54bda3, user_id=d20f5320adee461e9e90bc4465186913, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:02.203739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:02.241784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1136/2247] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:27:02 2022] PUT /identity/v3/projects/bb80a63667074f8f94417ad29b54bda3/users/d20f5320adee461e9e90bc4465186913/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:27:10.595394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b0db-f5f3-4782-b25a-83a71816ed72 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:10.596134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b0db-f5f3-4782-b25a-83a71816ed72 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:10.596134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b0db-f5f3-4782-b25a-83a71816ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:10.630343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bf35b0db-f5f3-4782-b25a-83a71816ed72 None None] MFA Rules not processed for user `d20f5320adee461e9e90bc4465186913`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:10.631635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bf35b0db-f5f3-4782-b25a-83a71816ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:10.784430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1112/2248] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:10.787932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:10.789822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:10.849236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:10.900227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a387b-18e2-4164-acfc-dd6b0e5239b9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:10.909344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:10.910097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:10.910216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:10.910458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:11.041681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:11.043248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e192447-05f4-4f5d-8d7c-ada50e5538f8 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:11.068594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1137/2249] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:20.766070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:21.054886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=41 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:22.059326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:22.059918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:22.061027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:22.093614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] MFA Rules not processed for user `b680411b142642b7acc3585d2481f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:22.094882 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:22.199244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1113/2250] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:29.701702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:29.702282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:29.702282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:29.743995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:29.745290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:29.808180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1138/2251] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:27:29.829041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:29.943018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dd426-0e80-4265-a8ad-31fa8d968ef5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:29.948442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:29.948982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:29.949350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:29.949525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:29.958890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:29.960630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:29.970132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1114/2252] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:27:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:27:32.791387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:32.792847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:32.862867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f2e9-3242-4c70-a9b3-2f0786805c46 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:32.867353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:32.868016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:32.868300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:32.868589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] PATH_INFO: `/v3/users/1a7627957f0c489ebf27dcab206542ef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:32.869762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7627957f0c489ebf27dcab206542ef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:32.871674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:32.931761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7627957f0c489ebf27dcab206542ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:32.932413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7627957f0c489ebf27dcab206542ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:32.933140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7627957f0c489ebf27dcab206542ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:32.945833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7627957f0c489ebf27dcab206542ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:32.952813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdc4b1bf-dc9e-40f9-a5bb-5aa32a2e7336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7627957f0c489ebf27dcab206542ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:32.968262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1139/2253] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:27:32 2022] DELETE /identity/v3/users/1a7627957f0c489ebf27dcab206542ef => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:27:33.156097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.167314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbaa5f-9787-4237-ba28-677fd92df86c service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.224387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.225078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.225359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.225646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] PATH_INFO: `/v3/projects/fcc945cdc83549c3833b0e39a9496e26` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.226802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc945cdc83549c3833b0e39a9496e26)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.228952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.267432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:33.277136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc945cdc83549c3833b0e39a9496e26', 'request_id': 'req-0a02d3cd-dfb0-4213-8220-6420d11aed39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:33.282967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc945cdc83549c3833b0e39a9496e26', 'request_id': 'req-0a02d3cd-dfb0-4213-8220-6420d11aed39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:33.288981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc945cdc83549c3833b0e39a9496e26', 'request_id': 'req-0a02d3cd-dfb0-4213-8220-6420d11aed39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:33.304548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] The token cache is being invalidate because project fcc945cdc83549c3833b0e39a9496e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:27:33.305646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:33.306466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a02d3cd-dfb0-4213-8220-6420d11aed39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:27:33.308185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1115/2254] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:27:33 2022] DELETE /identity/v3/projects/fcc945cdc83549c3833b0e39a9496e26 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:27:33.313505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.313859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.314144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.334170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:33.335521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:33.355049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:33.398246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1140/2255] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:27:33.406078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.406367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.406644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.426483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:33.427677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:33.436363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1116/2256] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:33.440605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.450391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ca94-8297-4e06-934c-2bccd557253b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.455510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.456164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.456447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.456723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.457452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:27:33.457757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.459367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.474252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1141/2257] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:27:33.483589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.496688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963f22b-5349-4360-86c1-2c696df7502f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.501892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.502645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.502993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.503329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.504108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.506085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.536334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1117/2258] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:27:33 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:27:33.540497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.550760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697b9d-17b7-4183-bc2b-dbb992b0fd1e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.614163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.614783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.615066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.615346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.616033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.617944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.639154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1142/2259] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:27:33 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:27:33.643316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.655043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb91f9-d1ee-4bb7-abf8-53c7662123b2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.659850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.660526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.660803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.661076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.661778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.663373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.678196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1118/2260] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:27:33.682105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.692585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4ff8-872f-4cfa-af37-fa510abab644 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.697141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.697745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.698026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.698308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.699004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.700634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f3c52-5e75-4430-96c1-48e3e8f467db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.714798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1143/2261] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:27:33.719060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.730085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798704-2ee4-4852-94e1-46fb103f40fc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:33.734729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:33.735518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.735804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.736110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] PATH_INFO: `/v3/projects/ffacb22f6ae64161b9d14941686a815d/users/2bc825c8fe3c4d579343de5ce18b47cd/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.755979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffacb22f6ae64161b9d14941686a815d, user_id=2bc825c8fe3c4d579343de5ce18b47cd, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:33.758745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:33.772438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1119/2262] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:27:33 2022] PUT /identity/v3/projects/ffacb22f6ae64161b9d14941686a815d/users/2bc825c8fe3c4d579343de5ce18b47cd/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:27:33.782700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:33.783044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:33.783393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:33.824255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] MFA Rules not processed for user `2bc825c8fe3c4d579343de5ce18b47cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:33.825477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:33.921912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1144/2263] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:33.940756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:33.942926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:34.016450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976f36-9014-4f91-94e9-d666277e2d5b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:34.020773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:34.021591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:34.021897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:34.022197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:34.032871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:34.037285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27466b96-8017-4330-81b0-9b4993df0fff service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:34.057651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1120/2264] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:27:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:38.866744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:38.868391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:38.906413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:38.953847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc6ab9-4607-4a6b-9bb2-65431b40acdc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:38.957497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:38.958132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:38.958414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:38.958691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:38.960500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:39.034915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:39.036406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8dd99-dab7-4556-a16a-9a8d7a3acd9c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:39.048475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1145/2265] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:27:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:39.159332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:39.159875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:39.160152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:39.188119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] MFA Rules not processed for user `84b748f30ebc4c5aa2af6ddcd86037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:39.189373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:39.268681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1121/2266] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:27:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:42.424647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d85b8-24eb-47d3-9eb0-52c63ccca2b7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:42.425227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d85b8-24eb-47d3-9eb0-52c63ccca2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:42.425227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d85b8-24eb-47d3-9eb0-52c63ccca2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:42.450165 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1a7d85b8-24eb-47d3-9eb0-52c63ccca2b7 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:42.452124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1a7d85b8-24eb-47d3-9eb0-52c63ccca2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:42.524998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1146/2267] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:42.546363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:42.548640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:42.560236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:42.675804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cc617-8c59-49dc-a6a3-8de9543f4138 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:42.680845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:42.681444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:42.681723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:42.681997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:42.691786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:42.693099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269011a-b670-4e89-9ca3-ebb17dcacb92 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:42.701884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1122/2268] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:27:52.517975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=66 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:52.697850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=28 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:01.405275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acbdd1-2f8d-490b-ab4a-5479524c96a7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:01.406148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acbdd1-2f8d-490b-ab4a-5479524c96a7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:01.406436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acbdd1-2f8d-490b-ab4a-5479524c96a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:01.432049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-55acbdd1-2f8d-490b-ab4a-5479524c96a7 None None] MFA Rules not processed for user `fd829e78e710440687678701fea3cf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:01.433448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-55acbdd1-2f8d-490b-ab4a-5479524c96a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:01.491574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1147/2269] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:04.599445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571390-ec4a-44ea-892b-382ad52ac26c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:04.599975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571390-ec4a-44ea-892b-382ad52ac26c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:04.600100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571390-ec4a-44ea-892b-382ad52ac26c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:04.601413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1123/2270] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:28:04.608507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2016-dee4-4920-a3a3-2641d1c182d1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:04.608949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2016-dee4-4920-a3a3-2641d1c182d1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:04.609042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2016-dee4-4920-a3a3-2641d1c182d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:04.632172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-72da2016-dee4-4920-a3a3-2641d1c182d1 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:04.634067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-72da2016-dee4-4920-a3a3-2641d1c182d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:04.764357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1148/2271] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:28:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:04.776260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09571390-ec4a-44ea-892b-382ad52ac26c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:04.788963 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09571390-ec4a-44ea-892b-382ad52ac26c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:04.793682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:04.794449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:04.794762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:04.795036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:04.805827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:04.807278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8311f1d-18f7-4cac-9383-89338b4bf028 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:04.819138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1124/2272] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:04.926567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:04.926819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:04.927053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:04.928774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1149/2273] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:28:04.934063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409bb8d-12a7-4624-82c7-3d9ca9e3a268 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:04.934682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409bb8d-12a7-4624-82c7-3d9ca9e3a268 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:04.935458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409bb8d-12a7-4624-82c7-3d9ca9e3a268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:04.956864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e409bb8d-12a7-4624-82c7-3d9ca9e3a268 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:04.957838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e409bb8d-12a7-4624-82c7-3d9ca9e3a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:04.984586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1125/2274] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:28:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:05.001171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:05.003993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:05.029294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf39ab3-b08a-488b-bfce-e96dc0ff52df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:05.034695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:05.035889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:05.036443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:05.037077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:05.066032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:05.069678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6774bfd-91d6-4282-88f8-3fae2eb5904c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:05.095878 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1150/2275] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:05.297724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bcf45-f83f-49bf-8970-7d9304f7ef5a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:05.298012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bcf45-f83f-49bf-8970-7d9304f7ef5a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:05.298291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bcf45-f83f-49bf-8970-7d9304f7ef5a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:05.299909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1126/2276] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:28:05.308573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:05.308743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:05.308982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:05.342441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:05.344121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:05.363182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1151/2277] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:05.376980 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bcf45-f83f-49bf-8970-7d9304f7ef5a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:05.391884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bcf45-f83f-49bf-8970-7d9304f7ef5a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:05.397033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:05.398061 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:05.398351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:05.398646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:05.414921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:05.416459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e75916-6a2c-4918-bcd2-2aa954a41a90 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:05.429597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1127/2278] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:12.602970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:12.604409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:12.692561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb9e2-4e61-4da8-8cfc-a8a425ca20c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:12.701479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:12.702765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:12.702765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:12.702765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:12.704601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:12.722830 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:12.724663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:12.737265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1152/2279] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:15.425056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=15 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:21.380860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:21.381429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:21.381429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:21.400264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:21.401508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:21.409020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1128/2280] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:21.430749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:21.442285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18287-a31f-4e5c-945e-4f722ee952b1 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:21.447153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:21.447744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:21.448051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:21.448328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:21.453204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:21.457881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:21.459501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be84119-469b-4f26-aab5-a4787051ff1c service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:21.467672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1153/2281] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:28:24.368097 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:24.436515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc38e3c-c1fb-463e-b388-da4904381107 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:24.441114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:24.441754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:24.442034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:24.442310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] PATH_INFO: `/v3/users/2bc825c8fe3c4d579343de5ce18b47cd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:24.443473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc825c8fe3c4d579343de5ce18b47cd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:24.445430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:24.497649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc825c8fe3c4d579343de5ce18b47cd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:24.500874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc825c8fe3c4d579343de5ce18b47cd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:24.501709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc825c8fe3c4d579343de5ce18b47cd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:24.516006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc825c8fe3c4d579343de5ce18b47cd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:24.523580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc825c8fe3c4d579343de5ce18b47cd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:24.539289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1129/2282] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:28:24 2022] DELETE /identity/v3/users/2bc825c8fe3c4d579343de5ce18b47cd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:28:24.544342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:24.544904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:24.544904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:24.585088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:24.586512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:24.691730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1154/2283] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:24.705883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:24.708158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:24.802098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3817553-0822-4d3a-ab12-4cc64550926b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:24.829223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:24.829786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:24.830078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:24.830354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:24.847316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:24.849263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18f58-6ff4-407e-9ec9-f562af828669 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:24.861695 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1130/2284] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:25.506492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:25.508611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:25.549959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afe2bf-62ec-4466-b5e7-ab37779c68fb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:25.550670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:25.551066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:25.551589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:25.551944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] PATH_INFO: `/v3/projects/ffacb22f6ae64161b9d14941686a815d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:25.553977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffacb22f6ae64161b9d14941686a815d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:25.555669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:25.621220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffacb22f6ae64161b9d14941686a815d', 'request_id': 'req-3afa5b36-ff2c-4532-9d80-c414def300fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:25.627717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffacb22f6ae64161b9d14941686a815d', 'request_id': 'req-3afa5b36-ff2c-4532-9d80-c414def300fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:25.636214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffacb22f6ae64161b9d14941686a815d', 'request_id': 'req-3afa5b36-ff2c-4532-9d80-c414def300fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:25.651038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] The token cache is being invalidate because project ffacb22f6ae64161b9d14941686a815d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:28:25.651466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:25.652327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3afa5b36-ff2c-4532-9d80-c414def300fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:25.653867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1155/2285] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:28:25 2022] DELETE /identity/v3/projects/ffacb22f6ae64161b9d14941686a815d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:28:25.658779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:25.659325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:25.659325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:25.679314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:25.680800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:25.693236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:25.758951 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1131/2286] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:25.768140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:25.768431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:25.768711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:25.789203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:25.791271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:25.807094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1156/2287] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:25.811873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:25.824251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0f7b1-b3b5-496e-955b-30e87c518827 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:25.830830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:25.831563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:25.831915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:25.832292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:25.833070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:28:25.833443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:25.835343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:25.847777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1132/2288] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:28:25.852603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:25.863237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd598a7-a9df-4915-a4bc-ce813ea00b1e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:25.874758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:25.875620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:25.876080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:25.876522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:25.877420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:25.879850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:25.910784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1157/2289] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:28:25 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:28:25.916329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:25.944952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc8a76-730d-49ad-b068-3865f2af766c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:26.025131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:26.025762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.026047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.026327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.027002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:26.029067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:26.086833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1133/2290] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:28:25 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:28:26.088428 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:26.100681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b044e-66ce-44bc-8c36-c0af783040cd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:26.106009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:26.106690 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.106973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.107245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.108550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:26.110240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:26.133719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1158/2291] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:28:26.138496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:26.152177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45cd00-96cb-4886-aa46-1afc7ffb2743 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:26.168729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:26.169372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.172510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.172836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.173602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:26.175483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adbf25-08ac-4a02-ad2f-79d54b873166 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:26.193854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1134/2292] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:28:26.198530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:26.210378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a127d-6809-4f26-9fd7-38b54208e955 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:26.225961 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:26.226999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.227354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.227813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] PATH_INFO: `/v3/projects/e375cd1a70564a5098613091ea263228/users/664a0a5780404fc8a7754124d871b9a5/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.260699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e375cd1a70564a5098613091ea263228, user_id=664a0a5780404fc8a7754124d871b9a5, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:26.264202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:26.291540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1159/2293] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:28:26 2022] PUT /identity/v3/projects/e375cd1a70564a5098613091ea263228/users/664a0a5780404fc8a7754124d871b9a5/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:28:26.293228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b376c1-3c11-48a0-be9c-a6194cafebc4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.293697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b376c1-3c11-48a0-be9c-a6194cafebc4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.293982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b376c1-3c11-48a0-be9c-a6194cafebc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.329456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b0b376c1-3c11-48a0-be9c-a6194cafebc4 None None] MFA Rules not processed for user `664a0a5780404fc8a7754124d871b9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:26.330805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b0b376c1-3c11-48a0-be9c-a6194cafebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:26.410173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1135/2294] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:26.422862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:26.424940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:26.440122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:26.498059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2ee34-f5ab-40a6-ad29-4e448143ff73 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:26.501886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:26.502582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:26.502860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:26.503289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:26.512595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:26.514714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:26.525012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1160/2295] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:27.956387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfd032-1fba-43e3-9056-e9c11351d157 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:27.956387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfd032-1fba-43e3-9056-e9c11351d157 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:27.956387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfd032-1fba-43e3-9056-e9c11351d157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:27.975367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-04dfd032-1fba-43e3-9056-e9c11351d157 None None] MFA Rules not processed for user `f20047857e8c4e36a031813729c09a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:27.976218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04dfd032-1fba-43e3-9056-e9c11351d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:27.982847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:28.079113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1136/2296] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:29.255145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:29.321538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202718b-adfd-4ca0-94ab-fcf3fc93ba41 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:29.326142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:29.326814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:29.327112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:29.328011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] PATH_INFO: `/v3/users/664a0a5780404fc8a7754124d871b9a5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:29.329180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664a0a5780404fc8a7754124d871b9a5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:29.331102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:29.386613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664a0a5780404fc8a7754124d871b9a5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:29.387182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664a0a5780404fc8a7754124d871b9a5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:29.387818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664a0a5780404fc8a7754124d871b9a5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:29.401527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664a0a5780404fc8a7754124d871b9a5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:29.409442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664a0a5780404fc8a7754124d871b9a5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:29.423216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1161/2297] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:28:29 2022] DELETE /identity/v3/users/664a0a5780404fc8a7754124d871b9a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:28:29.428082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:29.428663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:29.428896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:29.449269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:29.450618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:29.526487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1137/2298] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:29.536495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:29.537925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:29.591600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:29.624827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05c31-57ff-4751-9554-ca319874ccb3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:29.629075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:29.629800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:29.630183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:29.631119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:29.641859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:29.643422 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:29.654067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1162/2299] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:30.083779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:30.085536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:30.099719 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa3ae1-19a5-492b-ba64-2f8e112e9606 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:30.104724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:30.105384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:30.105668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:30.106766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] PATH_INFO: `/v3/projects/e375cd1a70564a5098613091ea263228` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:30.117769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e375cd1a70564a5098613091ea263228)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:30.120311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:30.172497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e375cd1a70564a5098613091ea263228', 'request_id': 'req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:30.178724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e375cd1a70564a5098613091ea263228', 'request_id': 'req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:30.185888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e375cd1a70564a5098613091ea263228', 'request_id': 'req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:30.201075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] The token cache is being invalidate because project e375cd1a70564a5098613091ea263228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:28:30.201370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:30.202009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a4de2b3a-57ba-45f1-b5a4-a10dc1cac930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:28:30.203672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1138/2300] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:28:30 2022] DELETE /identity/v3/projects/e375cd1a70564a5098613091ea263228 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:28:39.649470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:39.865553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:39.866955 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:39.878207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f320a34-6de0-4d7d-83fb-ed1b73f6678d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:39.926239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:39.926854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:39.927149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:39.927415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:39.929148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:39.990323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:39.991748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:40.001374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1163/2301] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:28:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:28:40.197741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:49.997222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=50 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:51.095381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fab6bd-1eaa-4200-ad3e-c768e425b95d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:51.095890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fab6bd-1eaa-4200-ad3e-c768e425b95d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:51.096013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fab6bd-1eaa-4200-ad3e-c768e425b95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:51.117043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06fab6bd-1eaa-4200-ad3e-c768e425b95d None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:51.118328 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06fab6bd-1eaa-4200-ad3e-c768e425b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:51.178032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1139/2302] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:51.190347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:51.191788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:51.259543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b4fb3-4df9-4fc9-a7b3-d2a9fe5543b1 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:51.264528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:51.265138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:51.265451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:51.265733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:51.274259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:51.275940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:51.284401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1164/2303] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:28:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:29:01.170342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:01.280868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=32 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:14.827835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:14.828402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:14.828402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:14.848239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] MFA Rules not processed for user `b680411b142642b7acc3585d2481f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:14.849572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:14.904608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1140/2304] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:29:24.897846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:34.542993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:34.544590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:34.619663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d73ace-adb2-4bcc-934e-79ab2a39d4b4 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:34.624152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:34.624817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:34.625102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:34.625379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] PATH_INFO: `/v3/users/f20047857e8c4e36a031813729c09a2b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:34.626547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20047857e8c4e36a031813729c09a2b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:34.629069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:34.674360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20047857e8c4e36a031813729c09a2b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:34.676640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20047857e8c4e36a031813729c09a2b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:34.676640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20047857e8c4e36a031813729c09a2b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:34.688409 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20047857e8c4e36a031813729c09a2b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:34.695768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-381840ec-10f9-4f04-a01a-3822d87aa225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20047857e8c4e36a031813729c09a2b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:34.707984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1165/2305] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:29:34 2022] DELETE /identity/v3/users/f20047857e8c4e36a031813729c09a2b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:29:34.888182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:34.898875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bd22c-ca89-4798-8ecf-8380116378b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:34.955208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:34.955846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:34.956212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:34.956490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] PATH_INFO: `/v3/projects/25fefc765ecb42728a941ddad185e082` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:34.957652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fefc765ecb42728a941ddad185e082)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:34.959567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:35.007949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fefc765ecb42728a941ddad185e082', 'request_id': 'req-6f585ffd-ddba-47a7-9de6-83340dd14347'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:35.013573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fefc765ecb42728a941ddad185e082', 'request_id': 'req-6f585ffd-ddba-47a7-9de6-83340dd14347'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:35.022360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fefc765ecb42728a941ddad185e082', 'request_id': 'req-6f585ffd-ddba-47a7-9de6-83340dd14347'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:35.037165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] The token cache is being invalidate because project 25fefc765ecb42728a941ddad185e082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:29:35.037453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:35.038095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f585ffd-ddba-47a7-9de6-83340dd14347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:35.039622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1141/2306] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:29:34 2022] DELETE /identity/v3/projects/25fefc765ecb42728a941ddad185e082 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:29:44.704557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:45.033598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:51.446136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:51.446658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:51.446757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:51.465807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] MFA Rules not processed for user `7213149649d249f2b2da3d6c7dac0a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:51.466999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:51.522586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1166/2307] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:29:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:29:52.200850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:52.201394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:52.201491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:52.220365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] MFA Rules not processed for user `84b748f30ebc4c5aa2af6ddcd86037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:52.221573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:52.278636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1142/2308] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:01.515343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:01.796694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:01.798067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:01.856315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51439fe6-f5d8-4613-b92d-bc00952917a3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:01.860203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:01.860818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:01.861099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:01.861778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:01.863136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:01.936270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:01.937709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607067e-8066-4fe2-a8a6-98cd47e1f85b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:01.949570 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1167/2309] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:02.272003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:03.136302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:03.146715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae5c2e-c203-47fe-97fc-dc10e61501e7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:03.150286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:03.150894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:03.151175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:03.151448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:03.153221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:03.166872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:03.168339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:03.179366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1143/2310] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:09.267132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:09.267616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:09.267803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:09.286633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] MFA Rules not processed for user `0ac9aa3a8d7a4383b9211cbac185a325`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:09.287833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:09.342774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1168/2311] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:30:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:13.174245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:15.809711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:15.811100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:15.825082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9de45-9cd7-46c0-894e-e390047c5913 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:15.830936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:15.831549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:15.831828 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:15.832124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:15.834344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:15.844879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:15.846441 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6a16d-3a25-4a97-a53e-06c9b94da14f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:15.858143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1144/2312] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:19.336349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=79 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:25.852426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:28.804856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:28.806504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:28.819478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01d37-2f70-4ee0-b7ac-e209787bf30d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:28.823324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:28.824235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:28.824660 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:28.825073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:28.827779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:28.900831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:28.902196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf9497-2cc5-49c8-8bca-918c8aa058f0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:28.911092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1169/2313] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:29.974429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfc3f-2d9d-4e9e-b7ef-db0b422d9970 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:29.975254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfc3f-2d9d-4e9e-b7ef-db0b422d9970 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:29.975254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfc3f-2d9d-4e9e-b7ef-db0b422d9970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:29.995637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e30bfc3f-2d9d-4e9e-b7ef-db0b422d9970 None None] MFA Rules not processed for user `b680411b142642b7acc3585d2481f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:29.996944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e30bfc3f-2d9d-4e9e-b7ef-db0b422d9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.003926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1145/2314] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:30:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:30.225282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668131-8fd4-4819-a76b-4ed4457a0e9d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.225901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668131-8fd4-4819-a76b-4ed4457a0e9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.226008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668131-8fd4-4819-a76b-4ed4457a0e9d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.228027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1170/2315] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:30:30.234096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f726e16-cac2-47f5-acd2-1576ba57e775 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.234386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f726e16-cac2-47f5-acd2-1576ba57e775 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.234671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f726e16-cac2-47f5-acd2-1576ba57e775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.257894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f726e16-cac2-47f5-acd2-1576ba57e775 None None] MFA Rules not processed for user `d48c3743968a4f3da87a246e8f5f6e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:30.259378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f726e16-cac2-47f5-acd2-1576ba57e775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.266317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1146/2316] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:30.281211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7668131-8fd4-4819-a76b-4ed4457a0e9d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:30.293523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7668131-8fd4-4819-a76b-4ed4457a0e9d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:30.297084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:30.297696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.297978 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.298252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.306452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:30.307885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:30.315929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1171/2317] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:30.411153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe6fe1-93a0-4dc2-87d7-40ecae15ef2b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.411555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe6fe1-93a0-4dc2-87d7-40ecae15ef2b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.411827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe6fe1-93a0-4dc2-87d7-40ecae15ef2b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.413557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1147/2318] 10.222.0.72 () {54 vars in 921 bytes} [Thu Aug 4 10:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 10:30:30.415159 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:30.419675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.419987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.420271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.424415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd622ee-94fd-4d71-b7b7-d2d33c35032e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:30.428476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:30.429083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.429360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.429629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.432049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.441073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] MFA Rules not processed for user `45bdd938403d46d6b7b026abd46c564d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:30.442272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.442400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:30.444151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:30.453122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1172/2319] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:30:30.461422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1148/2320] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Aug 4 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:30.472219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:30.480787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c395001-df59-48dd-bf3d-1475826da903 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:30.486442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:30.487133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.487489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.487832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.498938 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:30.500517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:30.513404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1173/2321] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:30.577323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:30.578737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.656139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c3373-6c59-4a69-9446-96642eb042eb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:30.661575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:30.662198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:30.662478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:30.662754 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:30.664673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:30.677860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:30.679517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:30.690520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1149/2322] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:32.417366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:32.440162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccca57-cb26-4620-84e9-acdf91782ce8 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:32.448901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:32.449880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:32.450315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:32.450624 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:32.453166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:32.465696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:32.467202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:32.475945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1174/2323] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:30:33.646068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:33.647678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:33.669445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e2514-185b-4317-a931-7e833072a0a7 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:33.676210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:33.676839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:33.677121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:33.677398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:33.680287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:33.694505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:33.696846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:33.710495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1150/2324] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:34.997910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:34.999393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:35.057152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b030ec5-9535-49f4-9e94-f62acf2b8aea service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:35.061045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:35.061601 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:35.061949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:35.062214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:35.063981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:35.076643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:35.078081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:35.089209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1175/2325] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:43.704170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=47 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:45.083177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=78 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:51.488236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:51.499804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93148f-e85a-4393-87ac-98ff06f3c256 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:51.505503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:51.506125 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:51.506403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:51.506675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:51.508469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:51.520247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:51.521842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:51.532298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1151/2326] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:56.981027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:56.991225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40072a-4a4f-40a3-ad20-37a4d35e7538 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:56.995251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:56.995843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:56.996147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:56.996418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:56.998220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:57.061828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:57.063264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:57.073481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1176/2327] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:00.271682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:00.298680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515a7d5-ba0e-4a29-9d57-3100f1b0f1da service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:00.304455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:00.305158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.305362 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.305847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.307861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:00.322763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:00.324748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:00.338647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1152/2328] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:00.349500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:00.351676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:00.380873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce89a3-e8db-4d63-aa6f-4e0c66287d61 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:00.386347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:00.386992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.387331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.387607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] PATH_INFO: `/v3/users/84b748f30ebc4c5aa2af6ddcd86037cd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.388807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b748f30ebc4c5aa2af6ddcd86037cd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:00.390770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:00.440705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b748f30ebc4c5aa2af6ddcd86037cd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.441357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b748f30ebc4c5aa2af6ddcd86037cd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.442035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b748f30ebc4c5aa2af6ddcd86037cd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.455532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b748f30ebc4c5aa2af6ddcd86037cd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.463332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e2263084-0d80-4da1-b369-36132da90bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b748f30ebc4c5aa2af6ddcd86037cd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.476584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1177/2329] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:00 2022] DELETE /identity/v3/users/84b748f30ebc4c5aa2af6ddcd86037cd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:00.654135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:00.666037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc41be-7221-4b11-9097-f031d89e17b3 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:00.725116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:00.725756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.726035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.726308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] PATH_INFO: `/v3/projects/2a801dbffe56474d96c2abea31fd2fdc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.727471 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a801dbffe56474d96c2abea31fd2fdc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:00.729407 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:00.776558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a801dbffe56474d96c2abea31fd2fdc', 'request_id': 'req-221210ff-617c-41f0-b839-1672589ce47f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.781849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a801dbffe56474d96c2abea31fd2fdc', 'request_id': 'req-221210ff-617c-41f0-b839-1672589ce47f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.788015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a801dbffe56474d96c2abea31fd2fdc', 'request_id': 'req-221210ff-617c-41f0-b839-1672589ce47f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.799947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] The token cache is being invalidate because project 2a801dbffe56474d96c2abea31fd2fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:00.800255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.800890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-221210ff-617c-41f0-b839-1672589ce47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:00.802823 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1153/2330] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:00 2022] DELETE /identity/v3/projects/2a801dbffe56474d96c2abea31fd2fdc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:00.809287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.809581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.809858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.829345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:00.830539 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:00.887218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1178/2331] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:00.895105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.895398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.895682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.917443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:00.918585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:00.926798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1154/2332] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:00.931005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:00.940586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaaf80-2de5-4640-a4d4-0427fa4e33fc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:00.945085 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:00.945689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.945969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.946241 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.946965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:31:00.947268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:00.948905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:00.950318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:00.965148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1179/2333] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:31:00.966213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:00.975403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db79a1-48a5-4b63-8e66-01dba4d2baae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:00.980724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:00.981324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:00.981608 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:00.981886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:00.982631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:00.984586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.011971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1155/2334] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:00 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:31:01.016239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.026672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa7de1-c97a-48a6-b046-b976977336ad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.084707 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.085436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.085818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.086196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.086953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.089589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.111062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1180/2335] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:01.115374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.126158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ef84-53a7-4ca5-8596-5d0eadcdac2b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.130822 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.131408 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.131688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.132046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.132684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.134269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.143896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1156/2336] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:01.147826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.157839 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be29fa-cca1-4133-9c21-d2a81400950c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.162600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.163198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.163480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.163752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.164568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.166646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362ea43-1b4b-4156-8947-fbf0b6319566 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.178581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1181/2337] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:01.183239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.193179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dc792-e829-4c43-95c2-f7b5e183ba89 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.197802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.198481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.198760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.199030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] PATH_INFO: `/v3/projects/0a9bc161a94747c3b10ce39a07a5b845/users/f6c5ad5ad1ca4d7183b3d41064626dc6/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.213356 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9bc161a94747c3b10ce39a07a5b845, user_id=f6c5ad5ad1ca4d7183b3d41064626dc6, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.216021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.224323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:01.230094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1157/2338] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:01 2022] PUT /identity/v3/projects/0a9bc161a94747c3b10ce39a07a5b845/users/f6c5ad5ad1ca4d7183b3d41064626dc6/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:01.240569 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.240925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.241305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.268929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] MFA Rules not processed for user `f6c5ad5ad1ca4d7183b3d41064626dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:01.270161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:01.332973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1182/2339] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:01.337487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.398182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8c828-63ee-4629-ba07-d1f0a640f598 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.403178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.403786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.404098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.404368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.405023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.406934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.434036 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1158/2340] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:31:01.438045 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.448552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5854-6a8c-444c-8444-3b93a672fc68 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.506066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.506777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.507063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.507339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.508038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.510077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.519005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:01.531975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1183/2341] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:01.536085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.547169 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b986d-b310-4d21-aa57-bb4dbc7209d6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.552107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.552721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.552998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.553271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.554132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.556516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.568506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1159/2342] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:01.572653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.583146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36236740-f8d8-48cb-a1eb-8913319e2cd3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.588588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.589382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.589780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.590174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] PATH_INFO: `/v3/projects/1517024ffb894ff7a50c1474a653925c/users/76ff498200334433b8f711334ce4e1b0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.608064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1517024ffb894ff7a50c1474a653925c, user_id=76ff498200334433b8f711334ce4e1b0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.611027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.625173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1184/2343] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:01 2022] PUT /identity/v3/projects/1517024ffb894ff7a50c1474a653925c/users/76ff498200334433b8f711334ce4e1b0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:01.629220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.639829 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94dd24-69a2-40e7-99d5-057722c4b1d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.698421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.699048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.699330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.699609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.700369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.701965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7c3dd-4afe-4e26-8e88-a3e2cbbecc62 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.711744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1160/2344] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:31:01.716351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.726520 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b691063-981c-411d-b933-fb51ab32f43f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.731253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.731909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.732223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.732495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] PATH_INFO: `/v3/projects/1517024ffb894ff7a50c1474a653925c/users/76ff498200334433b8f711334ce4e1b0/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.734402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1517024ffb894ff7a50c1474a653925c, user_id=76ff498200334433b8f711334ce4e1b0, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.737067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.747434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1185/2345] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:01 2022] PUT /identity/v3/projects/1517024ffb894ff7a50c1474a653925c/users/76ff498200334433b8f711334ce4e1b0/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:01.758578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.758886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.759723 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.786542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] MFA Rules not processed for user `76ff498200334433b8f711334ce4e1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:01.787657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:01.846142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:01.856864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1161/2346] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:01.867345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.869799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:01.924186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123a71-3ae1-4cb0-bfd6-af0fc564a682 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.928142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.928739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.929016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.929286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.938778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.940155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.949256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1186/2347] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:02.050054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:02.060412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fc9eb-497f-4edc-b176-af60d363a770 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:02.063996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:02.064605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:02.064889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:02.065162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:02.121724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:02.123179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036aa4f-00f0-437a-9ece-a185e761eb7d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:02.131892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1162/2348] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:04.718821 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:04.780001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c077c0-ae7e-48e6-9647-910d8cacf698 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:04.784465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:04.785092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:04.785370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:04.785640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] PATH_INFO: `/v3/users/f6c5ad5ad1ca4d7183b3d41064626dc6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:04.786805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c5ad5ad1ca4d7183b3d41064626dc6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:04.788722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:04.840126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c5ad5ad1ca4d7183b3d41064626dc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:04.840771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c5ad5ad1ca4d7183b3d41064626dc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:04.841371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c5ad5ad1ca4d7183b3d41064626dc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:04.848829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:04.854410 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c5ad5ad1ca4d7183b3d41064626dc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:04.861869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c5ad5ad1ca4d7183b3d41064626dc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:04.878918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1187/2349] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:04 2022] DELETE /identity/v3/users/f6c5ad5ad1ca4d7183b3d41064626dc6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:04.883478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:04.884016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:04.884126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:04.902848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:04.904074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:04.961228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1163/2350] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:04.970726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:04.972216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:05.028279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad07b-b5dc-4c0d-8d0a-c1473b4c0ece admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:05.031891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:05.032518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.032801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.033072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:05.042559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:05.044004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:05.053719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1188/2351] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:05.331622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:05.333303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:05.349003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a3a6-34a6-45eb-8ce9-ab56fc4ce4a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:05.354217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:05.354991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.355336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.355680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] PATH_INFO: `/v3/projects/0a9bc161a94747c3b10ce39a07a5b845` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:05.357834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9bc161a94747c3b10ce39a07a5b845)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:05.360110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:05.423555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9bc161a94747c3b10ce39a07a5b845', 'request_id': 'req-13439a17-dc62-4288-9183-f556731565dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.429487 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9bc161a94747c3b10ce39a07a5b845', 'request_id': 'req-13439a17-dc62-4288-9183-f556731565dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.435689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9bc161a94747c3b10ce39a07a5b845', 'request_id': 'req-13439a17-dc62-4288-9183-f556731565dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.450730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] The token cache is being invalidate because project 0a9bc161a94747c3b10ce39a07a5b845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:05.451022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.451840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.453436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1164/2352] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:05 2022] DELETE /identity/v3/projects/0a9bc161a94747c3b10ce39a07a5b845 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:05.457517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:05.467877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510032-be31-43f0-87ff-0abb408b8dae service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:05.524928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:05.525576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.525861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.526136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] PATH_INFO: `/v3/users/76ff498200334433b8f711334ce4e1b0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:05.527298 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ff498200334433b8f711334ce4e1b0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:05.529242 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:05.575719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ff498200334433b8f711334ce4e1b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.576454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ff498200334433b8f711334ce4e1b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.577063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ff498200334433b8f711334ce4e1b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.589755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ff498200334433b8f711334ce4e1b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.596537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c817c667-e6ab-453a-94e2-3877b2f100e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ff498200334433b8f711334ce4e1b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:05.605863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:05.612398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1189/2353] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:05 2022] DELETE /identity/v3/users/76ff498200334433b8f711334ce4e1b0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:05.909575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:05.912047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:05.931457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13439a17-dc62-4288-9183-f556731565dc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:05.946185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:06.000298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.000842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.001137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.001398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] PATH_INFO: `/v3/projects/1517024ffb894ff7a50c1474a653925c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.002563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1517024ffb894ff7a50c1474a653925c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.004522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.051314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1517024ffb894ff7a50c1474a653925c', 'request_id': 'req-12145ada-be4d-4fac-910d-a0f7b6f67509'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:06.057310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1517024ffb894ff7a50c1474a653925c', 'request_id': 'req-12145ada-be4d-4fac-910d-a0f7b6f67509'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:06.063286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1517024ffb894ff7a50c1474a653925c', 'request_id': 'req-12145ada-be4d-4fac-910d-a0f7b6f67509'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:06.077018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] The token cache is being invalidate because project 1517024ffb894ff7a50c1474a653925c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:06.077536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:06.078156 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-12145ada-be4d-4fac-910d-a0f7b6f67509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:06.079691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1165/2354] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:05 2022] DELETE /identity/v3/projects/1517024ffb894ff7a50c1474a653925c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:06.084591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.085090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.085186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.107342 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:06.108574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:06.164580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1190/2355] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:06.173092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.173381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.173690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.193520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:06.194732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:06.202937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1166/2356] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:06.207037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.217141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96cfc7-555c-4940-9027-dc9c12123197 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.221755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.222361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.222655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.222927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.223620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:31:06.223928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.225541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.237634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1191/2357] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:31:06.242220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.253566 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55063426-2b62-46e9-9f82-1da11b7ec4e6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.259139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.260286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.260567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.260839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.261516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.263415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.294795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1167/2358] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:06.299138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.309865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ee65-88fd-4f1a-a11a-76bace5823bf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.363708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.364327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.364691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.364963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.365609 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.367484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.388702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1192/2359] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:06.392946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.403473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad215b-efab-4adc-903b-abbc3aec9949 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.408116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.408721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.408999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.409269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.409955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.411543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.421818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1168/2360] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:06.425622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.436394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2d51-755c-4306-9454-df2eec10a109 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.440990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.441589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.441869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.442140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.442826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.444437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325063a2-638a-435c-97d6-3bf76b60f83e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.454744 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1193/2361] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:06.458952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.468616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0089-c06f-471e-83f1-6f051960300e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.473123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.473766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.474185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.474456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] PATH_INFO: `/v3/projects/a16f97702b044bd288e756215526125f/users/4ad367c80811464abfc7b23dba4d428e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.489096 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16f97702b044bd288e756215526125f, user_id=4ad367c80811464abfc7b23dba4d428e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.491838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.505339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1169/2362] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:06 2022] PUT /identity/v3/projects/a16f97702b044bd288e756215526125f/users/4ad367c80811464abfc7b23dba4d428e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:06.516783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.517068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.517340 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.544759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] MFA Rules not processed for user `4ad367c80811464abfc7b23dba4d428e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:06.545962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:06.605757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:06.620254 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1194/2363] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:06.621971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.667430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:06.686869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06e718-c495-49d5-ae1b-035637939cc0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.691662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.692573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.692573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.692973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.693611 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.695497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.723052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1170/2364] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:31:06.727059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.738173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ba5c5-f0ae-4d50-8d5e-bb8259327b45 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.794124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.794734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.795017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.795292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.795945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.797879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.818544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1195/2365] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:06 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:06.822659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.832895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f48bd-809e-4f7a-af5e-f6330e891697 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.837991 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.838602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.838885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.839160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.839852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.841482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.852157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1171/2366] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:06.857087 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.866966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa974bc-b308-41e0-89f4-48818a989748 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.871573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.872270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.872550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.872827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] PATH_INFO: `/v3/projects/e01ee8f63fb548dd93f1e15efb851771/users/32c2d746a27e40608ec4e97b46f1806a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.886910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01ee8f63fb548dd93f1e15efb851771, user_id=32c2d746a27e40608ec4e97b46f1806a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.889589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.902892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1196/2367] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:06 2022] PUT /identity/v3/projects/e01ee8f63fb548dd93f1e15efb851771/users/32c2d746a27e40608ec4e97b46f1806a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:06.906967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:06.917820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcb80c-319c-4196-9948-a4c374de8451 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:06.977598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:06.978918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:06.979196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:06.979469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.980186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.981771 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4d7c5-0a85-4a9d-bb96-51206310de1a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.991714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1172/2368] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:06.995862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:07.006780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0b07-5ce1-4a61-9478-34586f6c3f93 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:07.011452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:07.012169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:07.012455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:07.012705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] PATH_INFO: `/v3/projects/e01ee8f63fb548dd93f1e15efb851771/users/32c2d746a27e40608ec4e97b46f1806a/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:07.014636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01ee8f63fb548dd93f1e15efb851771, user_id=32c2d746a27e40608ec4e97b46f1806a, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:07.017311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:07.028771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1197/2369] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:06 2022] PUT /identity/v3/projects/e01ee8f63fb548dd93f1e15efb851771/users/32c2d746a27e40608ec4e97b46f1806a/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:07.039037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:07.039321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:07.039601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:07.067759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] MFA Rules not processed for user `32c2d746a27e40608ec4e97b46f1806a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:07.069093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:07.136837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1173/2370] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:07 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:07.148101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:07.150496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:07.207596 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90e40-a57a-4b83-a72c-bca9a5677661 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:07.211262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:07.211874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:07.212871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:07.213148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:07.222032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:07.223459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdddf1b-bdcb-498f-ad46-a05dc19c46d8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:07.232785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1198/2371] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:07 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:07.380792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:07.390749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172350b-4100-4c4e-8cb5-06deb697bc28 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:07.394300 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:07.394902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:07.395186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:07.395545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:07.434626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:07.454935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:07.456397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:07.465792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1174/2372] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:07 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:09.250233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:09.250233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:09.250233 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:09.270011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] MFA Rules not processed for user `fd829e78e710440687678701fea3cf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:09.271228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:09.318382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:09.331434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1199/2373] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:09.668175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:09.669749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:09.739337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02bd1-a07c-4a41-8005-c614bdf79154 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:09.742844 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:09.743982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:09.744506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:09.744838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:09.746301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:09.755580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:09.758311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d2134-3f3f-4fca-a37c-15ad22d43cfd service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:09.766825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1175/2374] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:16.077962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:16.144039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ebaaf-3666-4b0b-8738-661a543ea356 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:16.148484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:16.149121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:16.149397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:16.149669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] PATH_INFO: `/v3/users/4ad367c80811464abfc7b23dba4d428e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:16.150859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad367c80811464abfc7b23dba4d428e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:16.152780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:16.206042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad367c80811464abfc7b23dba4d428e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.206812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad367c80811464abfc7b23dba4d428e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.207507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad367c80811464abfc7b23dba4d428e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.221017 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad367c80811464abfc7b23dba4d428e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.227883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad367c80811464abfc7b23dba4d428e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.242032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1200/2375] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:16 2022] DELETE /identity/v3/users/4ad367c80811464abfc7b23dba4d428e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:16.248107 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:16.248981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:16.249983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:16.271414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:16.272636 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:16.333257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1176/2376] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:16.343500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:16.344944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:16.400962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5e76a-ec1b-4c76-9976-719572582c96 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:16.404603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:16.405210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:16.405491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:16.405764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:16.414645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:16.416729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:16.425967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1201/2377] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:16.693357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:16.694868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:16.708244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de57c87-2e3b-4844-8866-3154e6e539cc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:16.714983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:16.715872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:16.716392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:16.716804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] PATH_INFO: `/v3/projects/a16f97702b044bd288e756215526125f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:16.717930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16f97702b044bd288e756215526125f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:16.719891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:16.768263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16f97702b044bd288e756215526125f', 'request_id': 'req-2edab525-e360-434e-93f7-42852121b943'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.773747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16f97702b044bd288e756215526125f', 'request_id': 'req-2edab525-e360-434e-93f7-42852121b943'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.780560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16f97702b044bd288e756215526125f', 'request_id': 'req-2edab525-e360-434e-93f7-42852121b943'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.792454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] The token cache is being invalidate because project a16f97702b044bd288e756215526125f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:16.792739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.793347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.794840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1177/2378] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:16 2022] DELETE /identity/v3/projects/a16f97702b044bd288e756215526125f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:16.798928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:16.809235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452e7e-2956-4343-afe0-3b6c5f03d7cb service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:16.821235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:16.869580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:16.870225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:16.870506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:16.870783 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] PATH_INFO: `/v3/users/32c2d746a27e40608ec4e97b46f1806a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:16.871943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c2d746a27e40608ec4e97b46f1806a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:16.873916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:16.923115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c2d746a27e40608ec4e97b46f1806a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.923749 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c2d746a27e40608ec4e97b46f1806a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.924396 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c2d746a27e40608ec4e97b46f1806a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.936081 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c2d746a27e40608ec4e97b46f1806a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.942709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98403e1a-42ff-471d-b48f-3b778996abda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c2d746a27e40608ec4e97b46f1806a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:16.956059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1202/2379] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:16 2022] DELETE /identity/v3/users/32c2d746a27e40608ec4e97b46f1806a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:17.444899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:17.449502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:17.471385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:17.476508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edab525-e360-434e-93f7-42852121b943 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:17.544060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:17.544998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.545354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.545684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] PATH_INFO: `/v3/projects/e01ee8f63fb548dd93f1e15efb851771` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.547016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01ee8f63fb548dd93f1e15efb851771)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:17.549240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:17.617038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01ee8f63fb548dd93f1e15efb851771', 'request_id': 'req-08b79a81-ad1b-48a4-818c-3fd977bbc583'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:17.623360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01ee8f63fb548dd93f1e15efb851771', 'request_id': 'req-08b79a81-ad1b-48a4-818c-3fd977bbc583'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:17.630278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01ee8f63fb548dd93f1e15efb851771', 'request_id': 'req-08b79a81-ad1b-48a4-818c-3fd977bbc583'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:17.644246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] The token cache is being invalidate because project e01ee8f63fb548dd93f1e15efb851771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:17.644544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:17.645243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08b79a81-ad1b-48a4-818c-3fd977bbc583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:17.647038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1178/2380] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:17 2022] DELETE /identity/v3/projects/e01ee8f63fb548dd93f1e15efb851771 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:17.653300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341d5d8-4ef3-4888-964c-e1c96cb73280 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.653784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341d5d8-4ef3-4888-964c-e1c96cb73280 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.653902 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341d5d8-4ef3-4888-964c-e1c96cb73280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.675181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3341d5d8-4ef3-4888-964c-e1c96cb73280 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:17.676649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3341d5d8-4ef3-4888-964c-e1c96cb73280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:17.766055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1203/2381] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:17.774188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.774533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.774820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.793536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.793841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.794093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.804334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:17.805560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:17.825289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1179/2382] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:17.829543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:17.830140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:17.830789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:17.844590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d2edb-14af-4c60-ba39-912c6a7c99a7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:17.850789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:17.851405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.851689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.851995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.852694 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:31:17.853001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:17.877969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:17.882143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:17.907486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1180/2383] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:31:17.908514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:17.910413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:17.914149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1204/2384] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:17.928332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:17.962258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f1360-f62c-4be1-85d0-c070987ee370 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:17.971939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:17.972778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:17.973060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:17.973336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:17.974011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:17.975939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.008944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1205/2385] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:17 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:18.008944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.008944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:18.017392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61bb2-937e-4924-b667-0b623c580a7d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.027297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:18.079177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3a0e5-ab92-4071-9f53-719a616549dd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.082934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.083556 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.083833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.084251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.084555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.085177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.085460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.085739 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.086102 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:18.101077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.102781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.114111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1181/2386] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:18.115003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.148399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.150712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.159380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1206/2387] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:31:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:31:18.193847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3eda-9895-4aff-b349-21665884f291 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.199527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.200211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.200494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.200768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.201437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.203568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.231324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1182/2388] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:18 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:18.232662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.249246 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e224d-08a9-4a7b-a677-8ea886e8c631 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.259209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.261993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.262290 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.262566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.263300 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.266251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.278672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1207/2389] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:18.280267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.291496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff4f4-28bd-4e29-b578-5eec13208e01 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.299370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.300176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.300589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.300944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.302167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.303865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfbd9d-7388-4668-9f7c-04bae8d92a77 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.320049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1183/2390] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:31:18.324038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.336741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39969a-a857-4af2-87d6-eeca1ba14e61 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.344487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.345180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.345462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.345737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] PATH_INFO: `/v3/projects/cc88bc614dfb4b619ea4b6abbc2e385a/users/77b7fb485d384861a7ca98aa86d2dc4e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.378113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc88bc614dfb4b619ea4b6abbc2e385a, user_id=77b7fb485d384861a7ca98aa86d2dc4e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.381696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.420150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1208/2391] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:18 2022] PUT /identity/v3/projects/cc88bc614dfb4b619ea4b6abbc2e385a/users/77b7fb485d384861a7ca98aa86d2dc4e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:18.431780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.432117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.432361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.478173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] MFA Rules not processed for user `77b7fb485d384861a7ca98aa86d2dc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:18.480289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:18.556241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1184/2392] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:18.569429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:18.571379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:18.627216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:18.677245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54312b0-65ef-4f92-9848-ce4065fc2964 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:18.683073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:18.683771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:18.684109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:18.684619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:18.697433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:18.698888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46e6b4-590e-4d4a-a7e1-0c907eb45947 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:18.709344 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1209/2393] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:27.546190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:27.554868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:27.614684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82878337-f70e-4591-8662-c6682a8ab910 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:27.619113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:27.619750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:27.620057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:27.620324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] PATH_INFO: `/v3/users/77b7fb485d384861a7ca98aa86d2dc4e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:27.621499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b7fb485d384861a7ca98aa86d2dc4e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:27.623419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:27.669469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b7fb485d384861a7ca98aa86d2dc4e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:27.670046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b7fb485d384861a7ca98aa86d2dc4e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:27.670672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b7fb485d384861a7ca98aa86d2dc4e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:27.682682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b7fb485d384861a7ca98aa86d2dc4e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:27.689439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b7fb485d384861a7ca98aa86d2dc4e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:27.702913 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1185/2394] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:27 2022] DELETE /identity/v3/users/77b7fb485d384861a7ca98aa86d2dc4e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:31:27.707374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:27.707800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:27.707927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:27.727328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:27.728627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:27.787895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1210/2395] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:27.797954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:27.799354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:27.867271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4de72-f01d-497b-9e84-26072bb7eb2d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:27.871853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:27.872506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:27.872790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:27.873068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:27.883919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:27.886012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65e02d-3215-4d38-af93-8a370916564d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:27.899577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1186/2396] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:28.349936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.355353 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:28.389551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fb3b7-b1c8-4b0e-b9cd-d3c9c3962062 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.397402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.398222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.398511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.398786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] PATH_INFO: `/v3/projects/cc88bc614dfb4b619ea4b6abbc2e385a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.400122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc88bc614dfb4b619ea4b6abbc2e385a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.402473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.463896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc88bc614dfb4b619ea4b6abbc2e385a', 'request_id': 'req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:28.480932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc88bc614dfb4b619ea4b6abbc2e385a', 'request_id': 'req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:28.490957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc88bc614dfb4b619ea4b6abbc2e385a', 'request_id': 'req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:28.503665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] The token cache is being invalidate because project cc88bc614dfb4b619ea4b6abbc2e385a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:28.503982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:28.504639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7dc3a7d6-5e30-4a85-b182-8f7a4bc03a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:28.509579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1211/2397] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:28 2022] DELETE /identity/v3/projects/cc88bc614dfb4b619ea4b6abbc2e385a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:28.511321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.511742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.512110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.542526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:28.543747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:28.570499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:28.607631 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1187/2398] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:28.615419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.615709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.616013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.635220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:28.636465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:28.644315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1212/2399] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:28.648670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.659113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf06e3-5651-4075-b4cf-e49dc4deffa5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.663726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.664373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.664655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.664932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.665725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:31:28.666518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.668137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.679798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1188/2400] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:31:28.689556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.699145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05a71-18ed-4f60-a3a1-2526641667c5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.703841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.704496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.704777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.705052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.705754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.707672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.736006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1213/2401] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:28 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:28.739858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.750713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7772783-5af2-4d81-ba32-a4a192165e9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.805760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.806393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.806675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.806949 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.807603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.809533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.830051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1189/2402] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:28 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:28.834153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.844834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c9729-8059-4e87-9a57-faa5724b475e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.850128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.850735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.851013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.851282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.851993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.853666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.863357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1214/2403] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:28.867401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.877327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87c285-576c-4eef-b1a5-1b15347a7641 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.881852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.882447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.882751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.883025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.883712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.885821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60a3a5-53f8-420b-a371-da79c448949b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.896018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1190/2404] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:28.899968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:28.910310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05268a59-15c8-476b-ad61-451073c0f069 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:28.914876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:28.915525 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.916439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.916643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] PATH_INFO: `/v3/projects/ca63afbe614f4304a242ff0455eb5ae3/users/31877d98312742f7b67baca7a9ccc743/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.924759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:28.933804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca63afbe614f4304a242ff0455eb5ae3, user_id=31877d98312742f7b67baca7a9ccc743, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:28.936753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:28.949735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1215/2405] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:28 2022] PUT /identity/v3/projects/ca63afbe614f4304a242ff0455eb5ae3/users/31877d98312742f7b67baca7a9ccc743/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:28.961074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:28.961361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:28.961639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:28.989244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] MFA Rules not processed for user `31877d98312742f7b67baca7a9ccc743`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:28.990374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:29.052610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1191/2406] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:29.057598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.122543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5221d-493a-416b-af5d-4b3a797395ff admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.127183 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.127787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.128094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.128368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.129038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.130906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.160999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1216/2407] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:29.162442 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.173706 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b6c98-69c0-4408-ae04-b822b8fa1bba None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.194583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:29.233028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.233648 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.233930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.234202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.234955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.236815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.267285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1192/2408] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:29.271143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.281654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677de3b2-ef93-49bc-beff-773152054416 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.286279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.286886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.287172 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.287450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.288186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.289782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.300189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1217/2409] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:31:29.304346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.314572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54943d-eec5-4876-aaa0-a07e092b31e7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.319207 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.319806 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.320121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.320397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.321086 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.323299 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342653c7-e49d-43d0-9c48-d48dbbd7ebb2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.332937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1193/2410] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:29.336776 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.346731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b70f-beee-485b-baf0-06e3fc2049e6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.351321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.351980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.352267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.352538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] PATH_INFO: `/v3/projects/700f1ab41cb649a1931d877239442ce0/users/67ff8fcefffe45b8b640f74699a99257/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.366819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=700f1ab41cb649a1931d877239442ce0, user_id=67ff8fcefffe45b8b640f74699a99257, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.370006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.382738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1218/2411] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:29 2022] PUT /identity/v3/projects/700f1ab41cb649a1931d877239442ce0/users/67ff8fcefffe45b8b640f74699a99257/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:31:29.393330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.393620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.393911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.422182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] MFA Rules not processed for user `67ff8fcefffe45b8b640f74699a99257`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:29.423389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:29.490119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1194/2412] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:29.495180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.560184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d605-46c1-4e58-851d-15119b5511ea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.564960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.565645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.566000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.566279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.566948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.568884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.597052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1219/2413] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:29.600925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.614677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d16f8-77e8-47d1-8297-4b03b6dcb335 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.670644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.671258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.671540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.671836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.672535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.674513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.695137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1195/2414] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:29.699754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.709956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558dccf-90ab-42b5-add9-667c014c5a4d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.715003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.715602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.715881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.716186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.716865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.718476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.729620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1220/2415] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:31:29.733662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.743721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8890a2c-3644-4bcd-a48b-6452998ae7c2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.748528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.749135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.749420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.749795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] PATH_INFO: `/v3/projects/305f32efa03445df85e0b1c317c6bd7d/users/a94d3d5632064f16ade9667f90b60dcc/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.766517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305f32efa03445df85e0b1c317c6bd7d, user_id=a94d3d5632064f16ade9667f90b60dcc, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.769244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.778627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:29.785414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1196/2416] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:29 2022] PUT /identity/v3/projects/305f32efa03445df85e0b1c317c6bd7d/users/a94d3d5632064f16ade9667f90b60dcc/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:29.789627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.800536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a859-5c57-4ca4-908f-5216b827fd99 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.809400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:29.866801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.867540 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.867889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.868335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.869414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.871319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e2c6-5119-402c-8fe5-7699e4aeb0fc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.886109 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1221/2417] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:29.887271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:29.910343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b748c9-8a9e-46e7-aa5c-c7fc2bd8f561 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:29.916267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:29.916946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.917287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.917545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] PATH_INFO: `/v3/projects/305f32efa03445df85e0b1c317c6bd7d/users/a94d3d5632064f16ade9667f90b60dcc/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.920233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305f32efa03445df85e0b1c317c6bd7d, user_id=a94d3d5632064f16ade9667f90b60dcc, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:29.926034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:29.939590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1197/2418] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:29 2022] PUT /identity/v3/projects/305f32efa03445df85e0b1c317c6bd7d/users/a94d3d5632064f16ade9667f90b60dcc/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:29.950178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:29.950464 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:29.950763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:29.980413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] MFA Rules not processed for user `a94d3d5632064f16ade9667f90b60dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:29.981700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:30.051255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1222/2419] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:30.062966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:30.064918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:30.123871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f717e-56d7-4387-a769-0b1ddc1df531 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:30.127624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:30.128288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:30.128571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:30.128845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:30.184377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:30.185807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:30.194562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1198/2420] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:31.739557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:31.741026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:31.753035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be922acd-6363-4178-be1c-f1fd4530a5e8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:31.756714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:31.757325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:31.757608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:31.757875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:31.815488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:31.816969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f22392-5d2f-4b0b-9489-cdb57a4c6752 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:31.826016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1223/2421] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:32.460459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:32.468911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9228d-3cf1-4e0d-ad4f-db991e84234d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:32.472438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:32.473038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:32.473315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:32.473590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:32.482954 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:32.484398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:32.494679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1199/2422] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:35.068553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:35.069750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:35.069750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:35.092273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:35.093671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:35.125861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:35.162711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1224/2423] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:31:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:35.177684 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:35.179135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:35.267973 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3404a3-862f-4767-b875-80102c463ee0 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:35.273515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:35.274343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:35.274628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:35.274952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:35.284164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:35.285789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:35.297748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1200/2424] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:31:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:31:36.535482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:36.631097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f5364-60d7-4a18-99bd-f0cf77f1208b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:36.636825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:36.637478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:36.637754 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:36.638027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] PATH_INFO: `/v3/users/31877d98312742f7b67baca7a9ccc743` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:36.639398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31877d98312742f7b67baca7a9ccc743)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:36.641781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:36.711164 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:36.715290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:36.749486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:36.760891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31877d98312742f7b67baca7a9ccc743'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:36.763102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31877d98312742f7b67baca7a9ccc743'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:36.765968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31877d98312742f7b67baca7a9ccc743'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:36.782518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31877d98312742f7b67baca7a9ccc743'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:36.803111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-924ba365-6636-4f67-9d09-89ef916f4cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31877d98312742f7b67baca7a9ccc743'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:36.823571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76c9c-a9c0-4d9b-8f11-731d6af64170 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:36.830499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:36.831253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:36.831601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:36.832118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:36.832518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1225/2425] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:36 2022] DELETE /identity/v3/users/31877d98312742f7b67baca7a9ccc743 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:36.837564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:36.837857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:36.838301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:36.838974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:36.860676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:36.862414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:36.875015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1201/2426] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:31:36.879062 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:36.881281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:36.951488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:36.974546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1226/2427] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:36.986868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:36.988541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:37.074858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea273-f945-48cc-b213-0c997249e204 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:37.159023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:37.159811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:37.160267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:37.160630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:37.173942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:37.175843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:37.188947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1202/2428] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:37.681936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:37.683346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:37.702386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe42fbd-75ba-4967-8335-8004e60a4419 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:37.715758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:37.716693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:37.716966 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:37.717323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] PATH_INFO: `/v3/projects/ca63afbe614f4304a242ff0455eb5ae3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:37.719507 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca63afbe614f4304a242ff0455eb5ae3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:37.722212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:37.839689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca63afbe614f4304a242ff0455eb5ae3', 'request_id': 'req-d977b984-02ce-4feb-b72a-849781069824'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:37.849155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca63afbe614f4304a242ff0455eb5ae3', 'request_id': 'req-d977b984-02ce-4feb-b72a-849781069824'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:37.879079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca63afbe614f4304a242ff0455eb5ae3', 'request_id': 'req-d977b984-02ce-4feb-b72a-849781069824'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:37.898476 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] The token cache is being invalidate because project ca63afbe614f4304a242ff0455eb5ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:37.898766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:37.899735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:37.901629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1227/2429] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:37 2022] DELETE /identity/v3/projects/ca63afbe614f4304a242ff0455eb5ae3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:37.905680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:37.918278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3573acc-5910-4ef9-bec4-ee9b96f3294d service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:38.011646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:38.012359 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:38.012632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:38.012910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] PATH_INFO: `/v3/users/67ff8fcefffe45b8b640f74699a99257` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:38.014103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ff8fcefffe45b8b640f74699a99257)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:38.016093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:38.018729 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:38.094737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ff8fcefffe45b8b640f74699a99257'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.095466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ff8fcefffe45b8b640f74699a99257'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.096560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ff8fcefffe45b8b640f74699a99257'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.114578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ff8fcefffe45b8b640f74699a99257'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.125099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ff8fcefffe45b8b640f74699a99257'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.153187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1203/2430] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:37 2022] DELETE /identity/v3/users/67ff8fcefffe45b8b640f74699a99257 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:38.697619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:38.699234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:38.713528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977b984-02ce-4feb-b72a-849781069824 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:38.789861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:38.791318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:38.791611 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:38.791944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] PATH_INFO: `/v3/projects/700f1ab41cb649a1931d877239442ce0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:38.793158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=700f1ab41cb649a1931d877239442ce0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:38.795130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:38.861585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700f1ab41cb649a1931d877239442ce0', 'request_id': 'req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.867877 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700f1ab41cb649a1931d877239442ce0', 'request_id': 'req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.878292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700f1ab41cb649a1931d877239442ce0', 'request_id': 'req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.891391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] The token cache is being invalidate because project 700f1ab41cb649a1931d877239442ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:38.891671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.892356 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:38.893847 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1228/2431] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:38 2022] DELETE /identity/v3/projects/700f1ab41cb649a1931d877239442ce0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:38.897893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:38.899398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:38.913165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a751d-d5fe-40b6-8f9e-5546ef232ea4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:38.976891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:38.977616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:38.978461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:38.978784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] PATH_INFO: `/v3/users/a94d3d5632064f16ade9667f90b60dcc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:38.980034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94d3d5632064f16ade9667f90b60dcc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:38.982048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:39.037431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94d3d5632064f16ade9667f90b60dcc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.038209 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94d3d5632064f16ade9667f90b60dcc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.038990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94d3d5632064f16ade9667f90b60dcc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.055371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94d3d5632064f16ade9667f90b60dcc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.064932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b951e9ac-125e-475c-b368-d1066e229734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94d3d5632064f16ade9667f90b60dcc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.075314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:39.082403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1204/2432] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:31:38 2022] DELETE /identity/v3/users/a94d3d5632064f16ade9667f90b60dcc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:39.412682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:39.414150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:39.434441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e59c2-23ab-4f14-a47a-c20e175f1a9e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:39.448676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:39.532723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:39.533440 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:39.533731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:39.534005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] PATH_INFO: `/v3/projects/305f32efa03445df85e0b1c317c6bd7d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:39.535330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] RBAC: Authorizing `identity:delete_project(project_id=305f32efa03445df85e0b1c317c6bd7d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:39.537287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:39.595669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305f32efa03445df85e0b1c317c6bd7d', 'request_id': 'req-98bc267f-7e2c-4d4d-8063-574e4dc2009d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.602091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305f32efa03445df85e0b1c317c6bd7d', 'request_id': 'req-98bc267f-7e2c-4d4d-8063-574e4dc2009d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.609239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305f32efa03445df85e0b1c317c6bd7d', 'request_id': 'req-98bc267f-7e2c-4d4d-8063-574e4dc2009d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.621718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] The token cache is being invalidate because project 305f32efa03445df85e0b1c317c6bd7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:31:39.622078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.622705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98bc267f-7e2c-4d4d-8063-574e4dc2009d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:31:39.624281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1229/2433] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:31:39 2022] DELETE /identity/v3/projects/305f32efa03445df85e0b1c317c6bd7d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:39.629330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:39.629786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:39.629907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:39.654818 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:39.656281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:39.741948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1205/2434] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:39.751462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:39.752014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:39.752014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:39.781170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:39.782547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:39.797191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1230/2435] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:39.801187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:39.812405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd8475-4b45-40d3-9205-1395ba6597d2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:39.820678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:39.820678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:39.820678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:39.820678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:39.821158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:31:39.821158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:39.821597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:39.835384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1206/2436] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:31:39.840225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:39.851417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05590206-9123-4fae-85ac-0c679cbfed42 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:39.884670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:39.885552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:39.885975 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:39.886475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:39.887408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:39.891790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:39.936037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1231/2437] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:31:39 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:39.937534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:39.960747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec43c8-ec54-4cf0-b9c3-6507177bff62 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:40.032968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:40.033591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.033867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.034134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.034798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:40.037094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:40.063550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1207/2438] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:31:39 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:31:40.068088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:40.079557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884640-5b71-4588-9bff-9b2927f00a78 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:40.084306 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:40.084928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.085207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.085478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.086205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:40.088632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:40.101080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1232/2439] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:40.108815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:40.125496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf222f8b-7d3d-4ff4-bab7-0a79ea1336a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:40.130472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:40.131215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.131494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.132496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.133196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:40.135319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e8e7b-5200-407f-8af5-01be3b01a186 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:40.148005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1208/2440] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:31:40.152916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:40.163552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d252-4621-45aa-9023-2df2fe4f62aa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:40.169091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:40.169874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.170220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.170562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] PATH_INFO: `/v3/projects/2bfad04404af45ffbdfed9720f5a9723/users/4b53214ff25d4c339b15f25ddd809bae/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.187547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfad04404af45ffbdfed9720f5a9723, user_id=4b53214ff25d4c339b15f25ddd809bae, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:40.190883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:40.204991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1233/2441] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:31:40 2022] PUT /identity/v3/projects/2bfad04404af45ffbdfed9720f5a9723/users/4b53214ff25d4c339b15f25ddd809bae/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:31:40.215663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8b22-49cf-487d-9fb3-b3d580e85c1a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.215994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8b22-49cf-487d-9fb3-b3d580e85c1a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.216292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8b22-49cf-487d-9fb3-b3d580e85c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.261230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7abb8b22-49cf-487d-9fb3-b3d580e85c1a None None] MFA Rules not processed for user `4b53214ff25d4c339b15f25ddd809bae`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:40.262529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7abb8b22-49cf-487d-9fb3-b3d580e85c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:40.328693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:40.341925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1209/2442] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:31:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:40.352755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:40.355199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:40.408413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:40.414268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b0a5-0e70-402d-8fd6-452da3b09459 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:40.419454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:40.420564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:40.420848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:40.421274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:40.433556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:40.438542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:40.445400 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1234/2443] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:31:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:50.341085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:50.441119 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:56.858711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:56.859195 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:56.859453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:56.880351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:56.881557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:56.940933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1210/2444] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:31:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:56.954326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:56.955933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:57.026145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c08f4c-f800-4553-9683-1a320825d140 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:57.031861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:57.032503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:57.032790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:57.033065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:57.041602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:57.043161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdd931-a7b4-409c-9bdd-69749a133d89 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:57.051851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1235/2445] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:31:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:32:00.162510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:00.228019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc55551-cfa0-4556-82f2-830607dd1fb2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:00.233449 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:00.234103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:00.234382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:00.234657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] PATH_INFO: `/v3/users/4b53214ff25d4c339b15f25ddd809bae` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:00.235825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b53214ff25d4c339b15f25ddd809bae)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:00.238194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:00.292353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b53214ff25d4c339b15f25ddd809bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.293176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b53214ff25d4c339b15f25ddd809bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.293685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b53214ff25d4c339b15f25ddd809bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.308937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b53214ff25d4c339b15f25ddd809bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.315916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b53214ff25d4c339b15f25ddd809bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.329303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1211/2446] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:00 2022] DELETE /identity/v3/users/4b53214ff25d4c339b15f25ddd809bae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:00.333768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:00.334240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:00.334375 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:00.354880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:00.357022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:00.436212 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1236/2447] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:00.452862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:00.454374 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:00.481999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:00.512672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2806b86-4bed-4a92-abc1-d043ce859513 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:00.516877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:00.517497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:00.517778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:00.518051 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:00.526897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:00.528357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:00.538434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1212/2448] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:00.819527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:00.821040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:00.834055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb845fb-6e68-4685-be06-870f2d6ed978 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:00.838913 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:00.839571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:00.839852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:00.840237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] PATH_INFO: `/v3/projects/2bfad04404af45ffbdfed9720f5a9723` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:00.841705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfad04404af45ffbdfed9720f5a9723)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:00.843441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:00.890651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfad04404af45ffbdfed9720f5a9723', 'request_id': 'req-a9d1015f-af7d-41e3-95e7-84f2af028e46'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.896334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfad04404af45ffbdfed9720f5a9723', 'request_id': 'req-a9d1015f-af7d-41e3-95e7-84f2af028e46'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.903710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfad04404af45ffbdfed9720f5a9723', 'request_id': 'req-a9d1015f-af7d-41e3-95e7-84f2af028e46'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.914796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] The token cache is being invalidate because project 2bfad04404af45ffbdfed9720f5a9723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:00.915080 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.915697 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:00.917227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1237/2449] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:00 2022] DELETE /identity/v3/projects/2bfad04404af45ffbdfed9720f5a9723 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:05.668772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:05.670230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:05.681549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986babe-e382-44b4-bc93-ea63ccab40e1 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:05.731994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:05.732649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:05.732956 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:05.733224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:05.734964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:05.800707 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:05.802524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:05.814814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1213/2450] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:06.937798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:06.939245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:06.951402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1015f-af7d-41e3-95e7-84f2af028e46 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:06.955012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:06.955696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:06.956110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:06.956455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:06.958248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:06.970826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:06.972289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:06.977583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:06.985835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1238/2451] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:09.215273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:09.218082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:09.302738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:09.304833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:09.331562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999af3d-3273-4c01-b1c5-b4861b99e646 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:09.337068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b75f-75f3-41d6-9663-b7c388d42009 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:09.337273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:09.337925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.338486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.338949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] PATH_INFO: `/v3/users/b680411b142642b7acc3585d2481f880` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.340523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b680411b142642b7acc3585d2481f880)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:09.343274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:09.344916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:09.345908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.346197 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.346472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.349368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:09.366206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:09.367827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:09.382238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1214/2452] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:09.415114 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b680411b142642b7acc3585d2481f880'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.415911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b680411b142642b7acc3585d2481f880'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.416724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b680411b142642b7acc3585d2481f880'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.433195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b680411b142642b7acc3585d2481f880'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.444297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1556a89b-841f-449b-9b36-68e5d0cac710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b680411b142642b7acc3585d2481f880'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.459361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1239/2453] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:09 2022] DELETE /identity/v3/users/b680411b142642b7acc3585d2481f880 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:09.654181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:09.668383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee4ea-d88b-44f1-b606-cae20a0d4537 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:09.672341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:09.733574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:09.734217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.734496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.734772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] PATH_INFO: `/v3/projects/ae1e51ae816f4dc6b7dac8c2ddcf38bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.735920 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1e51ae816f4dc6b7dac8c2ddcf38bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:09.737945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:09.794219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1e51ae816f4dc6b7dac8c2ddcf38bc', 'request_id': 'req-2aa19146-9308-4381-9627-f99db9c4910c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.801470 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1e51ae816f4dc6b7dac8c2ddcf38bc', 'request_id': 'req-2aa19146-9308-4381-9627-f99db9c4910c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.808759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1e51ae816f4dc6b7dac8c2ddcf38bc', 'request_id': 'req-2aa19146-9308-4381-9627-f99db9c4910c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.825319 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] The token cache is being invalidate because project ae1e51ae816f4dc6b7dac8c2ddcf38bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:09.825605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.827038 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2aa19146-9308-4381-9627-f99db9c4910c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:09.832485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1215/2454] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:09 2022] DELETE /identity/v3/projects/ae1e51ae816f4dc6b7dac8c2ddcf38bc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:09.833912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.834207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.834489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.871401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:09.872678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:09.932542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1240/2455] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:09.941085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.941358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.941626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.962108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:09.963336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:09.972671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1216/2456] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:09.976897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:09.988357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dbb7f-774d-4b24-9264-c2c4aa1e9e63 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:09.993035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:09.993639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:09.993916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:09.994191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:09.994879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:09.995178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:09.997044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.012875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1241/2457] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:10.013911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.030886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228f6c-c670-45e2-b78c-289d0feb917d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.035979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.036615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.037592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.037592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.038540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.040559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.073158 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1217/2458] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:10 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:10.077513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.090752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7694d8b-6e2a-44a6-a4a3-c343e6ad4d24 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.169560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.170927 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.171224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.171499 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.172338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.174337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.184889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:10.217922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1242/2459] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:10 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:10.219256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.232323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504f047-0145-4949-bfb1-177460d61e7b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.240927 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.241545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.241664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.241914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.242627 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.249840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.263224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1218/2460] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:10.267657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.281964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8662e6-8654-49cc-bb89-bc87613c7dae admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.290180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.291105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.291394 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.291674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.292508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.294138 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec211f1-75c8-4dfd-81b2-6b54bf353d9d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.308479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1243/2461] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:10.310811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.322331 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d01837-75fd-401c-85e0-b0e2ea6ed2d0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.335133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.336126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.336564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.337391 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] PATH_INFO: `/v3/projects/5f2a9de9be3c47069cc4477cd5433c68/users/e3091633c8d44c36bfeed4a44e54c051/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.356863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2a9de9be3c47069cc4477cd5433c68, user_id=e3091633c8d44c36bfeed4a44e54c051, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.359598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.379825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1219/2462] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:10 2022] PUT /identity/v3/projects/5f2a9de9be3c47069cc4477cd5433c68/users/e3091633c8d44c36bfeed4a44e54c051/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:10.390752 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.392564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.392858 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.441670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] MFA Rules not processed for user `e3091633c8d44c36bfeed4a44e54c051`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:10.442940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:10.548511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1244/2463] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:10.549203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.589975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:10.641618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bd090-bbf3-4a84-a3ec-f02eace02dff admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.647456 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.648141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.648477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.648782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.649466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.651415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.693680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1220/2464] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:10 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:10.696313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.697078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.698343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:10.718249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2c1f-dc50-4bfd-868e-a053eb1ce1fc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.760364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211aa08-683b-4786-83b2-f4db0c2439c7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.765825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.766435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.766713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.766985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.767638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.769612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.776187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.776907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.777265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.777500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] PATH_INFO: `/v3/users/0ac9aa3a8d7a4383b9211cbac185a325` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.778811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac9aa3a8d7a4383b9211cbac185a325)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.781093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.791213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1221/2465] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:10 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:10.795552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.805308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c763-9df0-4743-b9ef-e5c7d59677cb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.810269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.811084 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.811321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.811597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.812354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.813932 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.826778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1222/2466] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:10.830824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.833420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac9aa3a8d7a4383b9211cbac185a325'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:10.834088 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac9aa3a8d7a4383b9211cbac185a325'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:10.834708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac9aa3a8d7a4383b9211cbac185a325'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:10.840592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda79ae-9389-453a-8725-22b0904a0e63 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:10.845396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:10.846044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:10.846323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:10.846595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] PATH_INFO: `/v3/projects/825952a37f3a418294b8e29c351dbf11/users/c56e484d866a425d8480b0faff22075d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:10.848102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac9aa3a8d7a4383b9211cbac185a325'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:10.852593 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:10.855787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac9aa3a8d7a4383b9211cbac185a325'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:10.863154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] RBAC: Authorizing `identity:create_grant(project_id=825952a37f3a418294b8e29c351dbf11, user_id=c56e484d866a425d8480b0faff22075d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:10.865881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:10.871599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1245/2467] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:10 2022] DELETE /identity/v3/users/0ac9aa3a8d7a4383b9211cbac185a325 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:10.880114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1223/2468] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:10 2022] PUT /identity/v3/projects/825952a37f3a418294b8e29c351dbf11/users/c56e484d866a425d8480b0faff22075d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:10.884007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:10.885513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:10.954869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6fdea-b6c8-4196-a084-a9620b7d3ef1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.014795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.015335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.015614 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.015886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.016605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.018194 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.028798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1246/2469] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:11.033414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.043805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d40f43-c6cf-408a-a004-4dd77160810e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.048526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.049590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.050265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.050545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.050820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] PATH_INFO: `/v3/projects/825952a37f3a418294b8e29c351dbf11/users/c56e484d866a425d8480b0faff22075d/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.051097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.053154 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=825952a37f3a418294b8e29c351dbf11, user_id=c56e484d866a425d8480b0faff22075d, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.055795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929a813-a484-4929-b3f8-0686c7765d64 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.066190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27801f6-79b0-4d09-b4f4-c51d491248c4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.071029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.071669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.071947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.072251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] PATH_INFO: `/v3/projects/888ab157217942abaf98b99e8eb827de` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.073405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=888ab157217942abaf98b99e8eb827de)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.075333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.081254 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1224/2470] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:11 2022] PUT /identity/v3/projects/825952a37f3a418294b8e29c351dbf11/users/c56e484d866a425d8480b0faff22075d/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:11.092621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.092919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.093192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.122655 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] MFA Rules not processed for user `c56e484d866a425d8480b0faff22075d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:11.124492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.126703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888ab157217942abaf98b99e8eb827de', 'request_id': 'req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:11.132160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888ab157217942abaf98b99e8eb827de', 'request_id': 'req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:11.140000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888ab157217942abaf98b99e8eb827de', 'request_id': 'req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:11.153204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] The token cache is being invalidate because project 888ab157217942abaf98b99e8eb827de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:11.153492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:11.154148 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb1cbf86-38ec-4abb-805f-43a5b312cfbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:11.155693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1247/2471] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:11 2022] DELETE /identity/v3/projects/888ab157217942abaf98b99e8eb827de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:11.160363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4ff97-a2bb-412a-a701-df649aa3fcc8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.160654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4ff97-a2bb-412a-a701-df649aa3fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.160937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4ff97-a2bb-412a-a701-df649aa3fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.173553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:11.181348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-54f4ff97-a2bb-412a-a701-df649aa3fcc8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:11.182555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54f4ff97-a2bb-412a-a701-df649aa3fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.191602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:11.198638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1225/2472] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:11.211709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.213325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.252345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1248/2473] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:11.260245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.260533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.260812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.271056 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a393a-7f9d-4c59-b5f8-98e9af37f609 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.274642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.275252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.275528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.275797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.281763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:11.282963 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.286275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.287698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.290870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1249/2474] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:11.295647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.297862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1226/2475] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:11.305238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a0eb-de09-4c9c-9917-97eaf51806fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.311532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.312227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.312478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.312751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.313453 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:11.313756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.315812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.330190 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1250/2476] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:32:11.334430 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.345854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03c00-8c36-4a34-9f56-31e87677418d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.350688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.351410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.352148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.352497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.353223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.355067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.377011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.384090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1227/2477] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:11.387136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a7a45-0de1-490e-85bb-ac1304783358 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.388439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.437350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.438129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.438221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.438498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.440373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.459520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee1338-be51-4064-8658-7085fd4b3893 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.464222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.464824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.465104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.465381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.466043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.468484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.493043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1228/2478] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:11.494109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.500804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.502282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.504497 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e9021-9533-4877-9e6b-ffe2965e7d65 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.509121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.509716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.510004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.510281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.510967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.511505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1251/2479] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:11.512595 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.523562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1229/2480] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:11.527715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.538585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d2bec-0974-4e67-ba62-3e6023aa2a29 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.543213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.543814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.544126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.544405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.545680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.547253 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217a260-7437-40ab-abe9-774cd05741e9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.557243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1252/2481] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:11.562390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.574004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc289e-0f6a-4dbf-83a1-95e4680fede7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.579143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.580106 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.580398 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.580675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] PATH_INFO: `/v3/projects/3ae7fd2f02d148d1b1ff060f241309dd/users/a7413a7f8b224af7a8ac30c0e9ff4de4/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.597661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae7fd2f02d148d1b1ff060f241309dd, user_id=a7413a7f8b224af7a8ac30c0e9ff4de4, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.600698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.619805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1230/2482] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:11 2022] PUT /identity/v3/projects/3ae7fd2f02d148d1b1ff060f241309dd/users/a7413a7f8b224af7a8ac30c0e9ff4de4/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:11.631773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.632168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.632714 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.665003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] MFA Rules not processed for user `a7413a7f8b224af7a8ac30c0e9ff4de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:11.666382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:11.691852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:11.730760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1253/2483] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:11.741567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:11.761884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:11.797163 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece31155-49a3-492d-a45a-8325c6300906 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:11.800701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:11.801308 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:11.801587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:11.801858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:11.810219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:11.812280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850490aa-377c-4a7d-9c81-a2d6ce9e471c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:11.820862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1231/2484] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:14.224684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:14.226151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:14.289616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad00e0-4a7c-49cb-a15e-c00576f60193 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:14.294131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:14.294760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:14.295041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:14.295339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] PATH_INFO: `/v3/users/e3091633c8d44c36bfeed4a44e54c051` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:14.296516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3091633c8d44c36bfeed4a44e54c051)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:14.298421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:14.344238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3091633c8d44c36bfeed4a44e54c051'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.344885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3091633c8d44c36bfeed4a44e54c051'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.345498 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3091633c8d44c36bfeed4a44e54c051'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.357147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3091633c8d44c36bfeed4a44e54c051'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.363756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3091633c8d44c36bfeed4a44e54c051'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.376703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1254/2485] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:14 2022] DELETE /identity/v3/users/e3091633c8d44c36bfeed4a44e54c051 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:14.381285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:14.381816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:14.381816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:14.400740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:14.401900 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:14.459134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1232/2486] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:14.468886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:14.470284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:14.522321 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:14.528013 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a4175-3926-4f3f-81c1-88784702b3b1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:14.533283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:14.534255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:14.534512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:14.535164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:14.545487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:14.546940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:14.557272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1255/2487] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:14.843085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:14.844620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:14.857589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b8cac-9254-41e5-9ae9-a86f2c471ce4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:14.863324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:14.864007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:14.864291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:14.864568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] PATH_INFO: `/v3/projects/5f2a9de9be3c47069cc4477cd5433c68` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:14.865746 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2a9de9be3c47069cc4477cd5433c68)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:14.867638 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:14.939941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2a9de9be3c47069cc4477cd5433c68', 'request_id': 'req-26d3f086-96f3-4057-9c50-422d0e7b72be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.946618 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2a9de9be3c47069cc4477cd5433c68', 'request_id': 'req-26d3f086-96f3-4057-9c50-422d0e7b72be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.953455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2a9de9be3c47069cc4477cd5433c68', 'request_id': 'req-26d3f086-96f3-4057-9c50-422d0e7b72be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.967220 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] The token cache is being invalidate because project 5f2a9de9be3c47069cc4477cd5433c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:14.967513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.968167 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:14.969698 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1233/2488] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:14 2022] DELETE /identity/v3/projects/5f2a9de9be3c47069cc4477cd5433c68 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:14.974215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:14.984198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c189e-2276-4b82-b011-bdb9ef874ae4 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.040514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.041197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.041478 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.041751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] PATH_INFO: `/v3/users/c56e484d866a425d8480b0faff22075d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.042968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56e484d866a425d8480b0faff22075d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.045043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.091654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56e484d866a425d8480b0faff22075d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.092363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56e484d866a425d8480b0faff22075d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.093003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56e484d866a425d8480b0faff22075d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.105639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56e484d866a425d8480b0faff22075d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.112371 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3af2160d-23e3-48fa-8290-ad997db45026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56e484d866a425d8480b0faff22075d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.125247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1256/2489] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:14 2022] DELETE /identity/v3/users/c56e484d866a425d8480b0faff22075d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:15.407938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.409473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:15.422941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f086-96f3-4057-9c50-422d0e7b72be admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.468515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:15.482989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.483637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.483921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.484222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] PATH_INFO: `/v3/projects/825952a37f3a418294b8e29c351dbf11` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.485376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=825952a37f3a418294b8e29c351dbf11)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.487296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.532446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825952a37f3a418294b8e29c351dbf11', 'request_id': 'req-2cedef94-737a-45fc-b017-7ccf6291e695'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.538176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825952a37f3a418294b8e29c351dbf11', 'request_id': 'req-2cedef94-737a-45fc-b017-7ccf6291e695'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.544285 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825952a37f3a418294b8e29c351dbf11', 'request_id': 'req-2cedef94-737a-45fc-b017-7ccf6291e695'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.557367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] The token cache is being invalidate because project 825952a37f3a418294b8e29c351dbf11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:15.557657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.558291 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2cedef94-737a-45fc-b017-7ccf6291e695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:15.560046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1234/2490] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:15 2022] DELETE /identity/v3/projects/825952a37f3a418294b8e29c351dbf11 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:15.564784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.565256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.565351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.585281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:15.586473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:15.644437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1257/2491] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:15.652429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.652716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.652994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.673103 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:15.674846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:15.683028 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1235/2492] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:15.687862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.704369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38731-ff26-43f9-9b8b-d4e024946d47 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.708922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.709613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.709905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.710179 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.710867 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:15.711168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.712794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.715157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:15.725935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1258/2493] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:15.730310 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.739643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a29dfb-f3ed-46b9-946f-39eaf58834cf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.744345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.744953 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.745230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.745527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.746186 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.748582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.776535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1236/2494] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:15 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:15.780708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.790767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a220cde-93eb-4bd0-a740-9f55d73b8b8b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.846057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.846665 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.846948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.847228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.847920 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.849854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.871086 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1259/2495] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:15 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:15.875659 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.886033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2698cd-fc1d-494b-9da6-ad3e28ba5113 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.891242 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.891841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.892146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.892422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.893105 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.894669 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.904691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1237/2496] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:15.908471 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.918769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d981c-47d7-4d84-a7be-2ceed6e7fec7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.923368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.923992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.924274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.924696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.925234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.926986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee8e39-5f66-4d51-8cad-b080f46e7efc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.938331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1260/2497] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:15.942484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:15.952994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36403-eae8-4692-a206-24c130ffde48 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:15.970562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:15.970562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:15.970562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:15.970562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] PATH_INFO: `/v3/projects/007f549fc48a4a3eb233d9a3c983defd/users/6fea0398e531444ea308c2e794e23208/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:15.973421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007f549fc48a4a3eb233d9a3c983defd, user_id=6fea0398e531444ea308c2e794e23208, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:15.976511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:15.990050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1238/2498] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:15 2022] PUT /identity/v3/projects/007f549fc48a4a3eb233d9a3c983defd/users/6fea0398e531444ea308c2e794e23208/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:16.000758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.001050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.001325 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.042898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] MFA Rules not processed for user `6fea0398e531444ea308c2e794e23208`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:16.044199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:16.110035 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1261/2499] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:16.114574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.178838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19555-68c0-4a6c-8463-cc65b2ebb522 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.183547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.184236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.184522 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.184802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.186121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.188049 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.208807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:16.220345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1239/2500] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:16 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:16.221374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.232974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf817-4e4a-4382-85f8-783d13e59113 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.294477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.295639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.295918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.296220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.296891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.298832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.308980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:16.322635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1262/2501] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:16 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:16.326654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.337162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c012-3f59-4d85-aca0-0302e8a67fb4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.342113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.342753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.343032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.343306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.344019 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.345598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.356403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1240/2502] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:16.361167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.371251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170a6d-1502-4655-8b9a-d5c90baefbd2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.375905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.376592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.376875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.377152 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] PATH_INFO: `/v3/projects/512a391921c14c50bf64b8d287c3b634/users/e07f44f6f2b94ac9b38d179f377c5a10/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.392676 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=512a391921c14c50bf64b8d287c3b634, user_id=e07f44f6f2b94ac9b38d179f377c5a10, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.395341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.409633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1263/2503] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:16 2022] PUT /identity/v3/projects/512a391921c14c50bf64b8d287c3b634/users/e07f44f6f2b94ac9b38d179f377c5a10/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:16.413773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.424461 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75011ee0-c494-4aff-ab37-77594c0b505d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.481134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.481672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.481952 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.482229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.482939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.484562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ea928-4dab-4b51-afb5-4446528a18a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.494630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1241/2504] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:16.499266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.510931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f705e-d28d-4b6d-9c36-ed9a0de8c65b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.516258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.516884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.517167 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.517484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] PATH_INFO: `/v3/projects/512a391921c14c50bf64b8d287c3b634/users/e07f44f6f2b94ac9b38d179f377c5a10/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.519414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512a391921c14c50bf64b8d287c3b634, user_id=e07f44f6f2b94ac9b38d179f377c5a10, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.522069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.542071 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1264/2505] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:16 2022] PUT /identity/v3/projects/512a391921c14c50bf64b8d287c3b634/users/e07f44f6f2b94ac9b38d179f377c5a10/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:16.554001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.554296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.554610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.586571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] MFA Rules not processed for user `e07f44f6f2b94ac9b38d179f377c5a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:16.587808 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:16.669447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1242/2506] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:16.682380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.684163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:16.741561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296a29-5eb8-4e1f-baed-792ab9c26c41 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.745093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.745711 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.745993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.746270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.755727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.757351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.766993 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1265/2507] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:16.855369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:16.856800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:16.869011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e254816-816d-494e-a312-0c469ace0b89 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:16.872628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:16.873297 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:16.873580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:16.873856 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:16.937402 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:16.938855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dcb7c-6b78-49b6-975c-f351cda210f3 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:16.957730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1243/2508] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:17.216320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:17.279640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad11cd-6534-4154-8d1e-dc92b91ffe29 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:17.284235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:17.284862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:17.285139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:17.285496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] PATH_INFO: `/v3/users/6fea0398e531444ea308c2e794e23208` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:17.286548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fea0398e531444ea308c2e794e23208)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:17.288481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:17.335613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fea0398e531444ea308c2e794e23208'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.336324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fea0398e531444ea308c2e794e23208'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.336956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fea0398e531444ea308c2e794e23208'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.347643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:17.350151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fea0398e531444ea308c2e794e23208'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.359819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fea0398e531444ea308c2e794e23208'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.372501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1266/2509] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:17 2022] DELETE /identity/v3/users/6fea0398e531444ea308c2e794e23208 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:17.377661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:17.377930 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:17.378208 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:17.391642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:17.398277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:17.399476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:17.458795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1244/2510] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:17.468603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:17.469982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:17.525431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a49f3d-b80b-48e2-a4cd-aa91f6d1d7bb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:17.529020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:17.529625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:17.529904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:17.530173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:17.539036 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:17.540492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:17.550205 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1267/2511] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:17.813370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:17.814832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:17.833249 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58743-4ffb-4185-ade6-6725f4975845 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:17.838795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:17.839429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:17.839697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:17.840058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] PATH_INFO: `/v3/projects/007f549fc48a4a3eb233d9a3c983defd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:17.841176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=007f549fc48a4a3eb233d9a3c983defd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:17.843069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:17.889272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007f549fc48a4a3eb233d9a3c983defd', 'request_id': 'req-06cf72aa-3ef1-451f-87f3-50ca09a201ca'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.894773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007f549fc48a4a3eb233d9a3c983defd', 'request_id': 'req-06cf72aa-3ef1-451f-87f3-50ca09a201ca'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.900935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007f549fc48a4a3eb233d9a3c983defd', 'request_id': 'req-06cf72aa-3ef1-451f-87f3-50ca09a201ca'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.913315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] The token cache is being invalidate because project 007f549fc48a4a3eb233d9a3c983defd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:17.913713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.914324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:17.915835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1245/2512] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:17 2022] DELETE /identity/v3/projects/007f549fc48a4a3eb233d9a3c983defd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:17.920370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:17.930115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9edb-c150-41ca-bafd-716c45051795 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:17.985288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:17.985933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:17.986211 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:17.986484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] PATH_INFO: `/v3/users/e07f44f6f2b94ac9b38d179f377c5a10` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:17.987648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07f44f6f2b94ac9b38d179f377c5a10)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:17.989600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.041360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07f44f6f2b94ac9b38d179f377c5a10'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.042176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07f44f6f2b94ac9b38d179f377c5a10'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.042912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07f44f6f2b94ac9b38d179f377c5a10'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.056947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07f44f6f2b94ac9b38d179f377c5a10'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.065678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a19d6e0-18f1-4789-809b-8c42de3769cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07f44f6f2b94ac9b38d179f377c5a10'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.076126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:18.081388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1268/2513] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:17 2022] DELETE /identity/v3/users/e07f44f6f2b94ac9b38d179f377c5a10 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:18.372333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.373908 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:18.391888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf72aa-3ef1-451f-87f3-50ca09a201ca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.453573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.454215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.454499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.454780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] PATH_INFO: `/v3/projects/512a391921c14c50bf64b8d287c3b634` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.456070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=512a391921c14c50bf64b8d287c3b634)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.457990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.493245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:18.506467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512a391921c14c50bf64b8d287c3b634', 'request_id': 'req-19e4a282-1ca1-48dc-8330-f7541dd7a27c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.511854 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512a391921c14c50bf64b8d287c3b634', 'request_id': 'req-19e4a282-1ca1-48dc-8330-f7541dd7a27c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.518253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512a391921c14c50bf64b8d287c3b634', 'request_id': 'req-19e4a282-1ca1-48dc-8330-f7541dd7a27c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.531454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] The token cache is being invalidate because project 512a391921c14c50bf64b8d287c3b634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:18.531740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.532381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19e4a282-1ca1-48dc-8330-f7541dd7a27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:18.533876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1246/2514] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:18 2022] DELETE /identity/v3/projects/512a391921c14c50bf64b8d287c3b634 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:18.538895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.539169 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.539444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.558794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:18.559988 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:18.627363 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1269/2515] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:18.635431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.635715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.636015 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.656547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:18.657763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:18.665651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1247/2516] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:18.669976 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.680113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d9043-b26c-4130-9bb6-a982a0cdd886 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.684818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.685954 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.686232 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.686502 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.687376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:18.687675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.689296 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.700746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1270/2517] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:32:18.705662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.715073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533fc38-9b8b-4c9d-a0b0-91ae8ec4a2b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.719814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.720438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.720714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.720985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.721646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.723513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.749873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1248/2518] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:18 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:18.753921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.764268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76fccd-f7e1-4dab-965a-9806af0715ed admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.818578 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.819181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.819458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.819725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.820398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.822285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.844450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1271/2519] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:18 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:18.848538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.859023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef026c-c7b9-46e3-ada6-b2f1bd99a09a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.863701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.864324 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.864598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.864866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.865565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.867143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.876674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1249/2520] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:18.880408 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.890203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94779a-7991-4fc9-8fc5-90e8fb5c50a0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.894796 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.895389 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.895669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.895941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.896656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.898349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44693-5b74-4cc1-85a8-4927a267fa94 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.909240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1272/2521] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:18.913240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:18.923594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffb98-5eea-4593-b99b-0951e1600be8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:18.928446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:18.929093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.929372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.929642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] PATH_INFO: `/v3/projects/0b79fd25826a4af3b1fa009c7c2afb5d/users/5b35092e0cab413880729821661fd0bb/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.943866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b79fd25826a4af3b1fa009c7c2afb5d, user_id=5b35092e0cab413880729821661fd0bb, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:18.946538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:18.959315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1250/2522] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:18 2022] PUT /identity/v3/projects/0b79fd25826a4af3b1fa009c7c2afb5d/users/5b35092e0cab413880729821661fd0bb/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:18.970373 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:18.970873 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:18.970973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:18.998248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] MFA Rules not processed for user `5b35092e0cab413880729821661fd0bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:18.999452 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:19.055772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:19.068105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1273/2523] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:19.070879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.148337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc46e-359a-4104-9d64-e464cc0c2f65 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.153479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.154280 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.155130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.155481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.156275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.158274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.194763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1251/2524] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:19 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:19.196198 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.208751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f23552-d1ba-4010-b1ea-c09ac42a5655 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.283475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.284146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.284433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.284710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.285417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.287365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.312343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1274/2525] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:19 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:19.313550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.324244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006083b-a662-48b6-96be-ce3a380361d9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.330191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.330868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.331205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.331475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.332246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.333897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.346238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1252/2526] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:19.350741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.361349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c56f4-d831-4518-91ec-5a3ee02a356a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.366033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.366702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.366985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.367263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] PATH_INFO: `/v3/projects/8a7c300d57984678bd006690af971f0b/users/26950dfe11054bf2b55218ef3d8ea0f6/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.387201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7c300d57984678bd006690af971f0b, user_id=26950dfe11054bf2b55218ef3d8ea0f6, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.390488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.404788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1275/2527] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:19 2022] PUT /identity/v3/projects/8a7c300d57984678bd006690af971f0b/users/26950dfe11054bf2b55218ef3d8ea0f6/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:19.409623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.422769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d255a8-9b3e-4752-84dd-2f446130fc55 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.460495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:19.490996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.491676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.491992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.492264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.492955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.494544 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf0cd2-fc51-410b-ba2a-d05f61b1157a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.504532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1253/2528] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:19.508732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.519007 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e386c4f-1f82-4c18-9171-e2a1df051c6a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.523694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.524473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.524828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.525178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] PATH_INFO: `/v3/projects/8a7c300d57984678bd006690af971f0b/users/26950dfe11054bf2b55218ef3d8ea0f6/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.527597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7c300d57984678bd006690af971f0b, user_id=26950dfe11054bf2b55218ef3d8ea0f6, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.530906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.544073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1276/2529] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:19 2022] PUT /identity/v3/projects/8a7c300d57984678bd006690af971f0b/users/26950dfe11054bf2b55218ef3d8ea0f6/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:19.556466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.556772 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.557048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.584769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] MFA Rules not processed for user `26950dfe11054bf2b55218ef3d8ea0f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:19.585975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:19.655067 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1254/2530] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:19.666106 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:19.668072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:19.728376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44580d37-f383-4bed-85cc-be10758d8bdd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:19.732369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:19.733470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:19.733750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:19.734020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:19.789964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:19.791406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408cd85-34d2-40a5-a924-05116bdbbe40 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:19.800929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1277/2531] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:20.268864 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:20.270535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:20.333587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7083909-23e9-44bc-900a-75263bdfb754 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:20.338066 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:20.338701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:20.338979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:20.339255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] PATH_INFO: `/v3/users/a7413a7f8b224af7a8ac30c0e9ff4de4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:20.340433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7413a7f8b224af7a8ac30c0e9ff4de4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:20.342836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:20.386670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7413a7f8b224af7a8ac30c0e9ff4de4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.387432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7413a7f8b224af7a8ac30c0e9ff4de4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.388072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7413a7f8b224af7a8ac30c0e9ff4de4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.393860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:20.402554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7413a7f8b224af7a8ac30c0e9ff4de4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.409873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7413a7f8b224af7a8ac30c0e9ff4de4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.422210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1255/2532] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:20 2022] DELETE /identity/v3/users/a7413a7f8b224af7a8ac30c0e9ff4de4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:20.426758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:20.427528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:20.427528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:20.435033 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:20.448141 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:20.449358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:20.518688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1278/2533] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:20.528465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:20.530484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:20.587334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182cdbb-740b-4cc4-9914-0f7e586247be admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:20.591048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:20.592273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:20.592568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:20.592842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:20.601811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:20.603244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae886c6-3b64-4741-a225-a679493b07cc service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:20.612629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1256/2534] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:20.887563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:20.890633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:20.904272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0354a2b-4c5b-42ef-85b3-1fc22691a8d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:20.908814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:20.909455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:20.909732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:20.910003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] PATH_INFO: `/v3/projects/3ae7fd2f02d148d1b1ff060f241309dd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:20.912959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae7fd2f02d148d1b1ff060f241309dd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:20.915201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:20.962570 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae7fd2f02d148d1b1ff060f241309dd', 'request_id': 'req-6ec6be40-a717-48db-8e5d-c4ceddf6973e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.968387 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae7fd2f02d148d1b1ff060f241309dd', 'request_id': 'req-6ec6be40-a717-48db-8e5d-c4ceddf6973e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.975618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae7fd2f02d148d1b1ff060f241309dd', 'request_id': 'req-6ec6be40-a717-48db-8e5d-c4ceddf6973e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.986990 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] The token cache is being invalidate because project 3ae7fd2f02d148d1b1ff060f241309dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:20.987299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.987934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ec6be40-a717-48db-8e5d-c4ceddf6973e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:20.989491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1279/2535] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:20 2022] DELETE /identity/v3/projects/3ae7fd2f02d148d1b1ff060f241309dd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:20.994463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:20.994750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:20.995026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.015634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:21.016867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:21.074594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1257/2536] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:21.082610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.082900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.083182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.104967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:21.106702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:21.115032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1280/2537] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:21.119224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.129210 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1853d9-40eb-4dc9-95f2-428dfb80be24 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.133792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.134396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.134681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.134955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.135647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:21.135969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.137562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.148740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1258/2538] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:21.152808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.162887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f50dd-02e2-4daa-b177-0e7fa582e2b0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.167365 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.168001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.168286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.168557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.169225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.171225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.198968 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1281/2539] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:21.203567 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.213766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c78d1d-eb1a-494e-83ff-26f16df61f50 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.273674 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.274296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.274571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.274840 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.275510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.277418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.286116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:21.304269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1259/2540] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:21.304889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.316124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd6a7-3f48-4754-ae81-99041a1ad926 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.320731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.321362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.321639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.321908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.322598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.324195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.334305 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1282/2541] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:21.338370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.349140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d895a-621b-41ec-ac8c-7c4630ec039f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.355182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.355793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.356099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.356373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.357063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.359160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53708381-9025-4189-aa3d-9c7fb3447f07 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.369672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1260/2542] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:21.374479 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.384733 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc846f-789d-4e4a-b245-c8ae07f27586 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.390134 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.390947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.391228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.391506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] PATH_INFO: `/v3/projects/5dbb39cc6c1d48409af021078a7f2e6e/users/2ac9d4c02ebc4c14a36ec3b52c88a38b/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.408067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbb39cc6c1d48409af021078a7f2e6e, user_id=2ac9d4c02ebc4c14a36ec3b52c88a38b, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.410829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.423982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1283/2543] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:21 2022] PUT /identity/v3/projects/5dbb39cc6c1d48409af021078a7f2e6e/users/2ac9d4c02ebc4c14a36ec3b52c88a38b/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:21.434804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.435091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.435426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.464610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] MFA Rules not processed for user `2ac9d4c02ebc4c14a36ec3b52c88a38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:21.465819 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:21.541420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1261/2544] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:21.546636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.574528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:21.613708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7f16f-e109-4beb-bc85-3b08fd3b1545 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.618339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.618960 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.619239 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.619510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.620235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.622139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.648467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1284/2545] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:21.652526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.663042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85dde-c290-471c-9432-d2856f50b2f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.718555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.719095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.719379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.719653 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.720367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.722256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.743278 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1262/2546] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:21.747360 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.757691 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061afb31-dc06-4245-93eb-5f79986fa8ab admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.762333 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.762935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.763213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.763484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.764204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.765798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.775948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1285/2547] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:21.780401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.791112 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c2d5-b180-4365-985a-e1b876ec1238 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.796397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.797058 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.797342 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.797623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] PATH_INFO: `/v3/projects/3556554152194e55892fd2a260d6c3cd/users/eac3e5dd5848488096bb24a88fbbcd13/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.813708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3556554152194e55892fd2a260d6c3cd, user_id=eac3e5dd5848488096bb24a88fbbcd13, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.817446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.831119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1263/2548] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:21 2022] PUT /identity/v3/projects/3556554152194e55892fd2a260d6c3cd/users/eac3e5dd5848488096bb24a88fbbcd13/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:21.835273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.846063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3e12-8c73-491a-8850-b9388b03ffc8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.902207 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.902868 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.903149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.903447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.904693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.906280 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722600-75dd-4839-ace6-7e68f24da505 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.915930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1286/2549] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:21.921281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:21.931652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783e1c-4ba2-4eb6-92b5-196d612a8ee9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:21.936592 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:21.937246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.937528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.937804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] PATH_INFO: `/v3/projects/3556554152194e55892fd2a260d6c3cd/users/eac3e5dd5848488096bb24a88fbbcd13/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.939757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3556554152194e55892fd2a260d6c3cd, user_id=eac3e5dd5848488096bb24a88fbbcd13, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:21.942482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:21.952715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1264/2550] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:21 2022] PUT /identity/v3/projects/3556554152194e55892fd2a260d6c3cd/users/eac3e5dd5848488096bb24a88fbbcd13/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:21.964249 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:21.964542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:21.964832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:21.994005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] MFA Rules not processed for user `eac3e5dd5848488096bb24a88fbbcd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:21.995136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:22.059829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1287/2551] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:22.071689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:22.073395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:22.100327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:22.131919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fc39-4c8f-4fbc-b9c8-0b0ae785ac79 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:22.135541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:22.136178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:22.136464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:22.136742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:22.146037 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:22.147458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c2bf3-1e21-48e4-b550-682a30dd3e2c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:22.156945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1265/2552] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:22.530146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:22.549545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:22.593494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94fce-7aaa-4b67-a8b3-2fde1d6b42ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:22.598018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:22.598657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:22.598943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:22.599220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] PATH_INFO: `/v3/users/2ac9d4c02ebc4c14a36ec3b52c88a38b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:22.600411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac9d4c02ebc4c14a36ec3b52c88a38b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:22.602324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:22.651274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac9d4c02ebc4c14a36ec3b52c88a38b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:22.652106 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac9d4c02ebc4c14a36ec3b52c88a38b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:22.652738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac9d4c02ebc4c14a36ec3b52c88a38b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:22.665771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac9d4c02ebc4c14a36ec3b52c88a38b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:22.673482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac9d4c02ebc4c14a36ec3b52c88a38b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:22.687151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1288/2553] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:22 2022] DELETE /identity/v3/users/2ac9d4c02ebc4c14a36ec3b52c88a38b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:22.693450 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:22.693979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:22.694132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:22.719085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:22.720329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:22.781014 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1266/2554] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:22.791025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:22.792481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:22.856762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b40d-2ff4-4fb6-b6b8-916a25876e23 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:22.860986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:22.861760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:22.862108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:22.862493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:22.872474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:22.874176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:22.885729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1289/2555] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:23.163101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:23.164616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:23.176660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6102c-65ea-436c-8db0-b79dadd9e998 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:23.181475 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:23.182134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:23.182396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:23.182667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] PATH_INFO: `/v3/projects/5dbb39cc6c1d48409af021078a7f2e6e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:23.183821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbb39cc6c1d48409af021078a7f2e6e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:23.186262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:23.233272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbb39cc6c1d48409af021078a7f2e6e', 'request_id': 'req-20883213-0772-44ee-a3b4-8f800043d8d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.248578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbb39cc6c1d48409af021078a7f2e6e', 'request_id': 'req-20883213-0772-44ee-a3b4-8f800043d8d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.255493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbb39cc6c1d48409af021078a7f2e6e', 'request_id': 'req-20883213-0772-44ee-a3b4-8f800043d8d4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.269675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] The token cache is being invalidate because project 5dbb39cc6c1d48409af021078a7f2e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:23.270098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.270602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.272191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1267/2556] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:23 2022] DELETE /identity/v3/projects/5dbb39cc6c1d48409af021078a7f2e6e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:23.276175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:23.285784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2ab42-7fb1-4ecc-a362-07f3ffffec7d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:23.324468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:23.345565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:23.346270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:23.346558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:23.346834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] PATH_INFO: `/v3/users/eac3e5dd5848488096bb24a88fbbcd13` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:23.348073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac3e5dd5848488096bb24a88fbbcd13)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:23.350041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:23.394824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac3e5dd5848488096bb24a88fbbcd13'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.395467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac3e5dd5848488096bb24a88fbbcd13'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.396105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac3e5dd5848488096bb24a88fbbcd13'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.409295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac3e5dd5848488096bb24a88fbbcd13'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.415985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cd79eb2-98ed-40aa-abac-0b586392c686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac3e5dd5848488096bb24a88fbbcd13'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.428544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1290/2557] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:23 2022] DELETE /identity/v3/users/eac3e5dd5848488096bb24a88fbbcd13 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:23.706241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:23.707739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:23.721887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883213-0772-44ee-a3b4-8f800043d8d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:23.890316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:23.903372 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:23.904046 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:23.904865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:23.905140 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] PATH_INFO: `/v3/projects/3556554152194e55892fd2a260d6c3cd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:23.906289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3556554152194e55892fd2a260d6c3cd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:23.908233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:23.956017 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3556554152194e55892fd2a260d6c3cd', 'request_id': 'req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.962510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3556554152194e55892fd2a260d6c3cd', 'request_id': 'req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.971375 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3556554152194e55892fd2a260d6c3cd', 'request_id': 'req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.986970 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] The token cache is being invalidate because project 3556554152194e55892fd2a260d6c3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:23.987261 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.987880 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:23.989440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1268/2558] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:23 2022] DELETE /identity/v3/projects/3556554152194e55892fd2a260d6c3cd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:23.994162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be257b-9a08-4af4-8839-d80640d606b4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:23.994736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be257b-9a08-4af4-8839-d80640d606b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:23.994736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be257b-9a08-4af4-8839-d80640d606b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.014044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-80be257b-9a08-4af4-8839-d80640d606b4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:24.015259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80be257b-9a08-4af4-8839-d80640d606b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:24.052845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.054290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:24.075128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1291/2559] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:24.082535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.082826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.083105 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.102790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:24.104265 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:24.112475 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1292/2560] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:24.115194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd43bc-b4d8-48a5-b9a1-2e708b646722 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.116700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.118889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.119506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.119783 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.120089 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.122307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:24.125987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f22a9-0873-4ad4-b6e4-34b90fe29378 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.130592 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.131177 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.131459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.131731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.132444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:24.132747 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.134327 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.145713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1293/2561] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:32:24.149950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.159705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb796e4-9519-47fd-9e72-5d0f33ddd9ba admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.164372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.164946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.165222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.165493 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.166132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.168060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.185104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.187354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.195734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1294/2562] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:24.197178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1269/2563] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:24.200811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.220511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:24.267943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36f501-aac5-4af2-a9ef-2d0d9d733db2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.272750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.273366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.273652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.273924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.274583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.276541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.297728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1295/2564] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/users => generated 278 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:24.301863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.312939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8af4e-15f6-4418-8e4f-f3c349618b4d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.318010 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.318642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.318958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.319192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.319888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.321583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.331874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1270/2565] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:24.335876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.346523 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672cf03-6044-41ad-9546-5df2dae7ccdb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.352427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.352708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.352985 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.353273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.354023 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.355767 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6764b0b-3762-45b4-af53-ed0ea9a6eba6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.366204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1296/2566] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:24.370569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.380634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc3e0-7edf-4a1f-af6d-02d294d08a19 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.385448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.386109 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.386394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.386672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] PATH_INFO: `/v3/projects/7dd73fbc2cc44a0ea1f2a58a63492e66/users/fa320680bf3d4543a07417d0844f176f/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.402580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd73fbc2cc44a0ea1f2a58a63492e66, user_id=fa320680bf3d4543a07417d0844f176f, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.405410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.408021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:24.420727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1271/2567] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:24 2022] PUT /identity/v3/projects/7dd73fbc2cc44a0ea1f2a58a63492e66/users/fa320680bf3d4543a07417d0844f176f/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:24.432689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.432930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.436163 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.466038 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] MFA Rules not processed for user `fa320680bf3d4543a07417d0844f176f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:24.467700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:24.540661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1297/2568] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:24.545529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.609546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0255d-64eb-4eef-a7f6-7f822ae39775 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.614143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.614745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.615024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.615296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.615948 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.617838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.644619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1272/2569] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:24.648751 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.660069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d44799-8613-4adf-9f49-680d1feee774 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.714351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.714957 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.715235 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.715510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.716204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.718214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.739777 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1298/2570] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:24.743931 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.754657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c6bf5-affd-47fe-a4be-f83e3c387d16 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.759136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.759736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.760055 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.760334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.761022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.762617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.773114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1273/2571] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:24.777322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.788542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94beec5-48c7-4b3c-88d7-ee354f2232f9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.793180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.793832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.794111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.794388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] PATH_INFO: `/v3/projects/caf98758850e4b28b6d9bb2e862b0194/users/58f915f1791346d7b5dbcc1b63ba544c/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.807258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:24.812427 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf98758850e4b28b6d9bb2e862b0194, user_id=58f915f1791346d7b5dbcc1b63ba544c, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.815129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.828829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1299/2572] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:24 2022] PUT /identity/v3/projects/caf98758850e4b28b6d9bb2e862b0194/users/58f915f1791346d7b5dbcc1b63ba544c/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:24.832897 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.842994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabcc52-bec9-47c2-bfee-8903c2426bd9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.898816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.899432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.899715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.900016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.900701 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.902281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbc7bb-6d6c-4881-8fab-35fe3c10ac5e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.911839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1274/2573] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:24.916089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:24.926209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b0c-dfd5-415f-b903-991e49ca62f1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:24.931077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:24.931835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.932147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.932420 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] PATH_INFO: `/v3/projects/caf98758850e4b28b6d9bb2e862b0194/users/58f915f1791346d7b5dbcc1b63ba544c/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.934347 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf98758850e4b28b6d9bb2e862b0194, user_id=58f915f1791346d7b5dbcc1b63ba544c, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:24.937517 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:24.947276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1300/2574] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:24 2022] PUT /identity/v3/projects/caf98758850e4b28b6d9bb2e862b0194/users/58f915f1791346d7b5dbcc1b63ba544c/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:24.957780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:24.958071 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:24.958351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:24.985170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] MFA Rules not processed for user `58f915f1791346d7b5dbcc1b63ba544c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:24.986366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:25.035600 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:25.054583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1275/2575] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:25.066137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:25.120688 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0936-ea36-4b3b-bc7f-1227ca563751 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:25.124332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:25.124935 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:25.125213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:25.125484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:25.134262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:25.135683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:25.145354 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1301/2576] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:25.230934 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:25.240862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9098ac5-ef35-47f0-8f68-36b464188e0f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:25.244738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:25.245351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:25.245629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:25.245902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:25.301316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:25.302983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797077ff-4454-4a2c-9cdb-ec797c168e7c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:25.312227 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1276/2577] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:29.102932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:29.104842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:29.174314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16d4ec-3664-48a1-965a-1993e022fb15 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:29.179323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:29.179980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:29.180278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:29.180556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] PATH_INFO: `/v3/users/5b35092e0cab413880729821661fd0bb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:29.181719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b35092e0cab413880729821661fd0bb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:29.183654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:29.231191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b35092e0cab413880729821661fd0bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.231850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b35092e0cab413880729821661fd0bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.232544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b35092e0cab413880729821661fd0bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.245667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b35092e0cab413880729821661fd0bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.257307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b35092e0cab413880729821661fd0bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.267403 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:29.274561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1302/2578] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:29 2022] DELETE /identity/v3/users/5b35092e0cab413880729821661fd0bb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:29.278903 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:29.279630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:29.279630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:29.299173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:29.300473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:29.362076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1277/2579] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:29.372406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:29.374240 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:29.431983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2df2a-ce66-43bb-b4da-5946d838a3c7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:29.436208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:29.436825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:29.437106 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:29.437388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:29.446461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:29.447894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:29.458647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1303/2580] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:29.750539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:29.752279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:29.766016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b58327-c69b-4c2f-91d5-8fbd102686e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:29.771469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:29.772201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:29.772483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:29.772802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] PATH_INFO: `/v3/projects/0b79fd25826a4af3b1fa009c7c2afb5d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:29.774113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b79fd25826a4af3b1fa009c7c2afb5d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:29.776275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:29.824275 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b79fd25826a4af3b1fa009c7c2afb5d', 'request_id': 'req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.836635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b79fd25826a4af3b1fa009c7c2afb5d', 'request_id': 'req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.843785 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b79fd25826a4af3b1fa009c7c2afb5d', 'request_id': 'req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.856443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] The token cache is being invalidate because project 0b79fd25826a4af3b1fa009c7c2afb5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:29.856730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.857383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.858917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1278/2581] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:29 2022] DELETE /identity/v3/projects/0b79fd25826a4af3b1fa009c7c2afb5d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:29.862865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:29.872503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf221c-c458-4ac9-af0a-c46a1ee7633f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:29.928567 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:29.929203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:29.929488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:29.929763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] PATH_INFO: `/v3/users/26950dfe11054bf2b55218ef3d8ea0f6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:29.930912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26950dfe11054bf2b55218ef3d8ea0f6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:29.932863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:29.976940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26950dfe11054bf2b55218ef3d8ea0f6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.977581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26950dfe11054bf2b55218ef3d8ea0f6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.978192 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26950dfe11054bf2b55218ef3d8ea0f6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.990287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26950dfe11054bf2b55218ef3d8ea0f6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:29.997025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7f098ad-217c-4adc-8a29-70940cd90da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26950dfe11054bf2b55218ef3d8ea0f6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.010111 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1304/2582] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:29 2022] DELETE /identity/v3/users/26950dfe11054bf2b55218ef3d8ea0f6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:30.295355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.296848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:30.309966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892f0b-bb1a-4907-b429-4a69d4b9b35f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.318717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:30.371874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.372586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.373492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.373768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] PATH_INFO: `/v3/projects/8a7c300d57984678bd006690af971f0b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.374902 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7c300d57984678bd006690af971f0b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.376832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.423134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7c300d57984678bd006690af971f0b', 'request_id': 'req-3cd1e923-ea4f-44e3-bb46-8028bdce892e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.428805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7c300d57984678bd006690af971f0b', 'request_id': 'req-3cd1e923-ea4f-44e3-bb46-8028bdce892e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.434747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7c300d57984678bd006690af971f0b', 'request_id': 'req-3cd1e923-ea4f-44e3-bb46-8028bdce892e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.449266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] The token cache is being invalidate because project 8a7c300d57984678bd006690af971f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:30.449553 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.450166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3cd1e923-ea4f-44e3-bb46-8028bdce892e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:30.451683 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1279/2583] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:30 2022] DELETE /identity/v3/projects/8a7c300d57984678bd006690af971f0b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:30.457631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.458127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.458225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.477635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:30.478819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:30.495579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:30.538919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1305/2584] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:30.547938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.548360 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.548643 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.567859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:30.569082 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:30.580189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1280/2585] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:30.581831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.592331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f6122-4597-4f49-acab-e39998788804 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.596871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.597583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.597824 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.598100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.598826 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:30.599125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.600750 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.614187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1306/2586] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:30.618361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.627482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a42f-3173-47d9-ae87-40600f660f0b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.632263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.632787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.633087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.633354 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.634016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.635876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.665494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1281/2587] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:30 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:30.671206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.682016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd13aa-12c4-42e3-b850-cf3a0426a6db admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.746817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.747454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.747787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.748132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.748799 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.750794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.774446 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1307/2588] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:30 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:30.778891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.791401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a832-3ff7-4d8e-943a-1cd780598d88 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.799476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.800594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.801085 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.801576 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.803141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.805868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.819476 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1282/2589] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:30.823504 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.834181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53546-559c-430a-a2e2-2dde2b6bd26e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.838786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.839395 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.839677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.839979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.840670 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.842909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22cb5-0b73-4bb3-a8a9-d813ce001d35 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.853528 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1308/2590] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:30.857616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:30.867388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b3607-5333-4991-abfb-aad314f8399d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:30.871784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:30.872465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.872742 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.873013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] PATH_INFO: `/v3/projects/c8715bff119d476e9523f37daf8eb7c3/users/269e277f32964bb6b6706eab4df89041/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.886867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8715bff119d476e9523f37daf8eb7c3, user_id=269e277f32964bb6b6706eab4df89041, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:30.890003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:30.902628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1283/2591] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:30 2022] PUT /identity/v3/projects/c8715bff119d476e9523f37daf8eb7c3/users/269e277f32964bb6b6706eab4df89041/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:30.916139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:30.916431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:30.916712 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:30.943603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] MFA Rules not processed for user `269e277f32964bb6b6706eab4df89041`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:30.944842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:31.006896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1309/2592] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:31.018509 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:31.020288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:31.061779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:31.078403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e37f90-9ec8-47c4-af51-38663a25064c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:31.081981 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:31.082591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:31.082893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:31.083160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:31.091259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:31.093334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b7eff-e37b-43c2-8a38-1744cba0f78c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:31.102378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1284/2593] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:34.098850 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:34.117719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:34.162455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08fc2-a864-453c-828f-8cc4fae37552 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:34.166831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:34.167505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.167794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.168102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] PATH_INFO: `/v3/users/269e277f32964bb6b6706eab4df89041` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.169289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269e277f32964bb6b6706eab4df89041)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:34.171202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:34.215672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269e277f32964bb6b6706eab4df89041'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.216343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269e277f32964bb6b6706eab4df89041'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.216950 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269e277f32964bb6b6706eab4df89041'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.228293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269e277f32964bb6b6706eab4df89041'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.234945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269e277f32964bb6b6706eab4df89041'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.249433 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1310/2594] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:34 2022] DELETE /identity/v3/users/269e277f32964bb6b6706eab4df89041 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:34.254135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.254529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.254799 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.273873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:34.275062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:34.332877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1285/2595] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:34.342469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:34.344116 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:34.401664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4e2a0-f72b-4722-bcb4-bcaf2bf76b02 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:34.405388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:34.405999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.406288 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.406561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.415562 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:34.416962 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e32741-2a7d-4578-af27-6d0069e570a4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:34.426313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1311/2596] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:34.699493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:34.700946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:34.714225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4b27c-b54d-4e02-856b-48c2e9d78892 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:34.720617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:34.721632 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.721977 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.722321 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] PATH_INFO: `/v3/projects/c8715bff119d476e9523f37daf8eb7c3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.724437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8715bff119d476e9523f37daf8eb7c3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:34.726367 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:34.779087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8715bff119d476e9523f37daf8eb7c3', 'request_id': 'req-92da4768-6d69-4f95-afbf-ce807e3d01f0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.785098 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8715bff119d476e9523f37daf8eb7c3', 'request_id': 'req-92da4768-6d69-4f95-afbf-ce807e3d01f0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.791901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8715bff119d476e9523f37daf8eb7c3', 'request_id': 'req-92da4768-6d69-4f95-afbf-ce807e3d01f0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.805602 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] The token cache is being invalidate because project c8715bff119d476e9523f37daf8eb7c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:34.805906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.806587 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-92da4768-6d69-4f95-afbf-ce807e3d01f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:34.808425 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1286/2597] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:34 2022] DELETE /identity/v3/projects/c8715bff119d476e9523f37daf8eb7c3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:34.813874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.814197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.814546 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.834524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:34.835722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:34.890418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:34.908874 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1312/2598] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:34.916745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.916994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.917274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.938221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:34.939435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:34.948087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1287/2599] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:34.952584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:34.967042 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5927d-368d-4a9c-ab85-3ae499832dfb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:34.971729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:34.972448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:34.972793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:34.973130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:34.973890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:34.974228 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:34.975812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:34.987818 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1313/2600] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:34.992293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.002781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f9053-9655-4399-9ab0-317356fce843 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.007937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.008577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.008867 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.009117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.009769 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.011662 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.038467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1288/2601] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:34 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:35.042641 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.053162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e08f-d9f6-47b0-a620-7f7d98f90cef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.108186 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.108840 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.109069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.109341 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.110006 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.112060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.133069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1314/2602] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:35.137172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.147830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1898348-de8f-450f-b877-753022cb5ce6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.152420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.153016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.153296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.153568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.154779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.156488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.166215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1289/2603] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:35.170273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.180230 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525b6a6-00ef-45d4-beec-784d456d7bde admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.184813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.185937 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.186215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.186484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.187168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.188815 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bc64-dffe-4ea5-b7eb-19b584b38fe3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.199649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1315/2604] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:35.203879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.213753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11746c-935c-4be2-adb0-9e2e6f6b7cb4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.220079 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.220720 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.221004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.221293 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] PATH_INFO: `/v3/projects/e81fc1a0c6fe42e490d276d19d8cc1c0/users/de6d383db02e46de92dea29671417a51/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.236558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81fc1a0c6fe42e490d276d19d8cc1c0, user_id=de6d383db02e46de92dea29671417a51, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.239219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.251571 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:35.257144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1290/2605] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:35 2022] PUT /identity/v3/projects/e81fc1a0c6fe42e490d276d19d8cc1c0/users/de6d383db02e46de92dea29671417a51/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:35.268382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.268674 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.268952 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.297779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] MFA Rules not processed for user `de6d383db02e46de92dea29671417a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:35.299009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:35.362328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1316/2606] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:35.374283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.375990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:35.436016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543e65-57ae-49f5-96ef-3a8aa7cded2c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.439650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.440336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.440624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.440890 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.449915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.451841 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5882d39-17ad-4903-8f62-2e8b36f183a9 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.463560 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1291/2607] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:40.328795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.330243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.374323 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:40.401627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160252f4-9436-4139-9b68-a260fb3371bb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.407150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.408055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.408417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.409133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] PATH_INFO: `/v3/users/fa320680bf3d4543a07417d0844f176f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.410040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa320680bf3d4543a07417d0844f176f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.412073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.488487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa320680bf3d4543a07417d0844f176f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:40.489565 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa320680bf3d4543a07417d0844f176f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:40.490070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa320680bf3d4543a07417d0844f176f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:40.505512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa320680bf3d4543a07417d0844f176f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:40.514542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa320680bf3d4543a07417d0844f176f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:40.544000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1317/2608] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:40 2022] DELETE /identity/v3/users/fa320680bf3d4543a07417d0844f176f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:40.548251 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.549640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.549944 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.572108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:40.573496 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.646999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1292/2609] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:40.658404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.659970 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.723381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f53be-784a-4aa7-b026-31153bcff390 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.727435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.728122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.728405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.728678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.739413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.740880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.751319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1318/2610] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:41.082524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:41.085697 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:41.109244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4449-47cb-455f-8a84-d7973b30e934 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.127382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.128080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.128401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.128660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] PATH_INFO: `/v3/projects/7dd73fbc2cc44a0ea1f2a58a63492e66` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.130073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd73fbc2cc44a0ea1f2a58a63492e66)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.132160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.214428 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd73fbc2cc44a0ea1f2a58a63492e66', 'request_id': 'req-c377675e-c37d-4024-919d-cdcd529a24bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.221365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd73fbc2cc44a0ea1f2a58a63492e66', 'request_id': 'req-c377675e-c37d-4024-919d-cdcd529a24bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.230467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd73fbc2cc44a0ea1f2a58a63492e66', 'request_id': 'req-c377675e-c37d-4024-919d-cdcd529a24bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.245862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] The token cache is being invalidate because project 7dd73fbc2cc44a0ea1f2a58a63492e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:41.246155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.246965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.251875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1293/2611] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:41 2022] DELETE /identity/v3/projects/7dd73fbc2cc44a0ea1f2a58a63492e66 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:41.253201 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:41.268334 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478da4-91a8-455c-a796-474c8eee0408 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.321989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:41.328559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.329555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.330137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.330137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] PATH_INFO: `/v3/users/58f915f1791346d7b5dbcc1b63ba544c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.331374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f915f1791346d7b5dbcc1b63ba544c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.333860 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.391505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f915f1791346d7b5dbcc1b63ba544c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.392189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f915f1791346d7b5dbcc1b63ba544c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.392809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f915f1791346d7b5dbcc1b63ba544c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.404576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f915f1791346d7b5dbcc1b63ba544c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.411319 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d42003ef-9ece-4ea4-be79-e8cf6da08635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f915f1791346d7b5dbcc1b63ba544c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.425906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1319/2612] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:41 2022] DELETE /identity/v3/users/58f915f1791346d7b5dbcc1b63ba544c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:41.729110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:41.730755 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:41.745486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377675e-c37d-4024-919d-cdcd529a24bf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.775131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:41.811176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.812039 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.812323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.812667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] PATH_INFO: `/v3/projects/caf98758850e4b28b6d9bb2e862b0194` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.814052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf98758850e4b28b6d9bb2e862b0194)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.816011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.861800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf98758850e4b28b6d9bb2e862b0194', 'request_id': 'req-d6a86325-d9d0-4283-8044-c16bbd41978e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.867546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf98758850e4b28b6d9bb2e862b0194', 'request_id': 'req-d6a86325-d9d0-4283-8044-c16bbd41978e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.874460 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf98758850e4b28b6d9bb2e862b0194', 'request_id': 'req-d6a86325-d9d0-4283-8044-c16bbd41978e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.886642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] The token cache is being invalidate because project caf98758850e4b28b6d9bb2e862b0194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:41.886924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.887607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d6a86325-d9d0-4283-8044-c16bbd41978e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.889792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1294/2613] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:41 2022] DELETE /identity/v3/projects/caf98758850e4b28b6d9bb2e862b0194 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:41.894764 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.895057 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.895335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.915413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:41.916769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:41.976616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1320/2614] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:41.984667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.985021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.985392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.006262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:42.007573 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:42.016732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1295/2615] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:42.021162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.032040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440a922-a8c3-4eb0-b94c-7fa2553ee417 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.036946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.037576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.037857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.038129 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.038862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:42.039174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.040851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.053980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1321/2616] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:42.058433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.068545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c047b-5e33-40eb-bbe2-587cfeb99ffb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.073168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.073765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.074043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.074311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.074965 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.076923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.105040 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1296/2617] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:42.110037 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.120310 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512fce-7bd1-44db-b6a8-98571204c53c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.178393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.178903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.179181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.179450 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.180137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.182031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.192753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:42.208817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1322/2618] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:42.210060 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.220370 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77436-20ee-4d1b-85c5-c4c61da25b5e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.224979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.225583 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.225859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.226131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.226850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.228454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.245881 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1297/2619] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:42.251021 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.261594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc7948-7e98-4a1a-a4e2-6e4a3a0da970 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.266883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.267584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.267931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.268307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.269094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.270675 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d394-5e21-4068-958b-5993c810e5b4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.282140 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1323/2620] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:42.286415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.296904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0783fb-795d-46d9-8b89-29ea5a9586f6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.301477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.302122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.302403 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.303073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] PATH_INFO: `/v3/projects/39112f67800c41f99b3f9500d7c20441/users/fcb8306e2fb04f8aa09eac272050410f/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.317837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39112f67800c41f99b3f9500d7c20441, user_id=fcb8306e2fb04f8aa09eac272050410f, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.320504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.333872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1298/2621] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:42 2022] PUT /identity/v3/projects/39112f67800c41f99b3f9500d7c20441/users/fcb8306e2fb04f8aa09eac272050410f/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:42.345014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.345568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.345568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.375431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] MFA Rules not processed for user `fcb8306e2fb04f8aa09eac272050410f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:42.376694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:42.439043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1324/2622] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:42.443630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.496539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:42.507099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc718c-e9c1-484f-84fe-867057c112b9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.512502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.513108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.513381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.513647 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.514305 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.516199 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.541552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1299/2623] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:42.546746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.557655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c53c6c-5af0-4747-8760-c25f57dcdbe6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.615368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.616009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.616287 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.616556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.617261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.619196 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.640468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1325/2624] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:42.644665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.655349 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923039-f3d1-45d0-bec1-eb238c1a5251 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.660119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.660714 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.660994 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.661268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.661995 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.663892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.675143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1300/2625] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:42.681374 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.692787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721475a6-9411-4231-9e8c-eccc76b57712 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.697215 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.697871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.698145 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.698414 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] PATH_INFO: `/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe/users/187b158164be44399c11511f64ee71e1/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.714829 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4c2033cf1c4191a4d6fca8d210d2fe, user_id=187b158164be44399c11511f64ee71e1, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.718122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.731336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1326/2626] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:42 2022] PUT /identity/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe/users/187b158164be44399c11511f64ee71e1/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:42.736134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.746267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503354-a868-47a6-a918-526936820e90 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.809065 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.809679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.809960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.810256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.810940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.812535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca960da-498c-48e4-a65a-3d38ce94dbda admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.822325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1301/2627] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:42.827174 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.837587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f73bc-60d6-4517-8c0a-11ac33f10cb0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.851161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.851872 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.852397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.852669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] PATH_INFO: `/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe/users/187b158164be44399c11511f64ee71e1/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.854887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4c2033cf1c4191a4d6fca8d210d2fe, user_id=187b158164be44399c11511f64ee71e1, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.858566 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.869914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1327/2628] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:42 2022] PUT /identity/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe/users/187b158164be44399c11511f64ee71e1/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:42.883585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.884263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.884263 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.911525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] MFA Rules not processed for user `187b158164be44399c11511f64ee71e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:42.912767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.006873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1302/2629] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:43.018556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:43.020887 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.049238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:43.079646 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca65be-f11e-40e8-b5f9-fbb31734396a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:43.083359 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:43.083973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.084282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.084551 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.093651 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:43.095123 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.105613 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1328/2630] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:43.348993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:43.350437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.363023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12c6d1-3bd5-4fa0-8789-208862ba1b6f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:43.366892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:43.367498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.367843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.368203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.411688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:43.424899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:43.426326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.435394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1303/2631] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:44.267819 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:44.269289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:44.327618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f011a4-2217-461c-b0fb-7b60bc0dfc8e service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:44.331175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:44.331810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:44.332120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:44.332423 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:44.334173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:44.396189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:44.397759 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:44.408716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1329/2632] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:47.800439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:47.801985 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:47.816465 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e398bf7-2697-42eb-821d-544b0685a358 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:47.820992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:47.821635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.821918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.822222 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] PATH_INFO: `/v3/users/7213149649d249f2b2da3d6c7dac0a68` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.823397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7213149649d249f2b2da3d6c7dac0a68)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:47.825353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:47.877196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7213149649d249f2b2da3d6c7dac0a68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.877923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7213149649d249f2b2da3d6c7dac0a68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.878514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7213149649d249f2b2da3d6c7dac0a68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.891034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7213149649d249f2b2da3d6c7dac0a68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.898078 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d99f9c4-6521-4d58-9ac0-bf30fcf733ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7213149649d249f2b2da3d6c7dac0a68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.911715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1304/2633] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:47 2022] DELETE /identity/v3/users/7213149649d249f2b2da3d6c7dac0a68 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:48.102763 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.114685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f468d-fad7-4236-9a89-5d8de5760e61 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.170281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.170915 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.171195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.171467 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] PATH_INFO: `/v3/projects/35576cef915b41419fc74b0beea0f988` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.172630 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35576cef915b41419fc74b0beea0f988)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.174514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.195127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:48.222346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35576cef915b41419fc74b0beea0f988', 'request_id': 'req-1897a371-6631-43b9-aa11-60ad08094d33'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.228049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35576cef915b41419fc74b0beea0f988', 'request_id': 'req-1897a371-6631-43b9-aa11-60ad08094d33'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.240795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35576cef915b41419fc74b0beea0f988', 'request_id': 'req-1897a371-6631-43b9-aa11-60ad08094d33'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.251898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] The token cache is being invalidate because project 35576cef915b41419fc74b0beea0f988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:48.252278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.252931 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1897a371-6631-43b9-aa11-60ad08094d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.254698 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1330/2634] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:48 2022] DELETE /identity/v3/projects/35576cef915b41419fc74b0beea0f988 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:48.259516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.259809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.260108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.281858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.283012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.353795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1305/2635] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:48.362149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.362432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.362708 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.387485 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.396718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.398576 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.398750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.411115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1331/2636] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:48.413076 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88db8e-5382-450a-aa9f-f23d53347845 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.415041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.417891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.418523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.418801 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.419073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] PATH_INFO: `/v3/users/fcb8306e2fb04f8aa09eac272050410f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.420529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb8306e2fb04f8aa09eac272050410f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.422454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.424531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714e20-563d-4bed-b585-0f2523e79aa6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.437683 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.438996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.439281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.439702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.440732 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:48.441191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.443761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.459187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1332/2637] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:32:48.463881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.474534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2f86-4a95-42dd-8555-9b0b1a33127d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.477679 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb8306e2fb04f8aa09eac272050410f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.478414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb8306e2fb04f8aa09eac272050410f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.479025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb8306e2fb04f8aa09eac272050410f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.479881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.480621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.480907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.481184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.481943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.484009 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.485053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:48.497216 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb8306e2fb04f8aa09eac272050410f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.505503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e56479c-224d-426b-8eda-357ee0671632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb8306e2fb04f8aa09eac272050410f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:48.519448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1333/2638] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:48.519656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1306/2639] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:48 2022] DELETE /identity/v3/users/fcb8306e2fb04f8aa09eac272050410f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:48.523147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.523797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.524264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.524569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.524861 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.546166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.547279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.591900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319ede0-06a8-4a45-8963-d0bced0df949 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.633798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1307/2640] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:48.645223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.647063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.672173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.672841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.673127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.673762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.674191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.676143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.698229 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1334/2641] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:48.702793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.707727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:48.712260 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa892e4f-3ec8-43ab-b34c-81305f2465d7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.713512 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd5934-361f-47c0-bfb6-509200682b68 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.716248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.716866 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.717141 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.717409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.720225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.720841 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.721318 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.721672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.722678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.724297 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.726789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.728250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.734256 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1335/2642] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:48.738078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.739624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1308/2643] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:48.748180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e73efe-934a-444c-95b6-2b55205c4b96 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.752945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.753668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.754107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.754425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.755181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.756870 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9eac24-5ea5-41dd-a4a1-acef1ba22ed8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.767876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1336/2644] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:48.772256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.781879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf75c-0518-49d5-b076-25d58c77b919 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.787190 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.787847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.788165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.788439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] PATH_INFO: `/v3/projects/aacf5efd0b274d9484e0f08416404ff6/users/a79b35b536b7442b8e6e2b19583c40a9/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.805217 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacf5efd0b274d9484e0f08416404ff6, user_id=a79b35b536b7442b8e6e2b19583c40a9, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.808230 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.822198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1309/2645] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:48 2022] PUT /identity/v3/projects/aacf5efd0b274d9484e0f08416404ff6/users/a79b35b536b7442b8e6e2b19583c40a9/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:48.832882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.833170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.833447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.862187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] MFA Rules not processed for user `a79b35b536b7442b8e6e2b19583c40a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.863434 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.938583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1337/2646] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:48.943123 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.993955 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.008062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7dec-197a-46e8-a139-2b38f6fd39cd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.014540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.015256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.015842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.015842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.016203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.016492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.017619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.018495 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.032209 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4969ed3-5e4d-4e57-abc4-1a47c3e03766 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.037168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.037835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.038108 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.038384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] PATH_INFO: `/v3/projects/39112f67800c41f99b3f9500d7c20441` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.039709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39112f67800c41f99b3f9500d7c20441)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.041693 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.050181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1310/2647] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:49.055113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.102418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39112f67800c41f99b3f9500d7c20441', 'request_id': 'req-14457a2c-54fa-4ca1-9411-5bd7fed95696'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.112188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39112f67800c41f99b3f9500d7c20441', 'request_id': 'req-14457a2c-54fa-4ca1-9411-5bd7fed95696'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.119753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39112f67800c41f99b3f9500d7c20441', 'request_id': 'req-14457a2c-54fa-4ca1-9411-5bd7fed95696'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.119938 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989de69e-0eed-4287-b248-83919baf13a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.131770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] The token cache is being invalidate because project 39112f67800c41f99b3f9500d7c20441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:49.132093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.132756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.134270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1338/2648] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:49 2022] DELETE /identity/v3/projects/39112f67800c41f99b3f9500d7c20441 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:49.138237 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.140020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.178827 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.179437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.179721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.180021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.180681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.182590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.192629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.204177 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1311/2649] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:49 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:49.205516 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457a2c-54fa-4ca1-9411-5bd7fed95696 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.207833 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.209533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.211076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.211721 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.212214 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.212669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] PATH_INFO: `/v3/users/187b158164be44399c11511f64ee71e1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.214094 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] RBAC: Authorizing `identity:delete_user(user_id=187b158164be44399c11511f64ee71e1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.216026 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.222054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de8c1f-fc01-406b-94ba-d2b0715a7898 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.226989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.227671 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.227989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.228287 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.229108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.230687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.240835 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1312/2650] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:49.245205 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.257221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49b518-179f-4983-b992-95f1761d9997 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.262366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.263118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.263388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.263663 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] PATH_INFO: `/v3/projects/a6da0052aeca4fe4beef4e09be41043e/users/68a18c3117634e929fe9f1b20091c466/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.272538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187b158164be44399c11511f64ee71e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.273291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187b158164be44399c11511f64ee71e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.273889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187b158164be44399c11511f64ee71e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.279739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6da0052aeca4fe4beef4e09be41043e, user_id=68a18c3117634e929fe9f1b20091c466, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.282119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.286273 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187b158164be44399c11511f64ee71e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.293281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187b158164be44399c11511f64ee71e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.294629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1313/2651] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:49 2022] PUT /identity/v3/projects/a6da0052aeca4fe4beef4e09be41043e/users/68a18c3117634e929fe9f1b20091c466/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:49.298545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.300204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.307930 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1339/2652] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:49 2022] DELETE /identity/v3/users/187b158164be44399c11511f64ee71e1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:49.342968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.367568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a7f3-6f3a-4a2b-8324-82acf6fdcc69 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.424502 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.425181 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.425474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.425753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.426484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.428369 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2cf3ec-fdef-42a7-b9de-2dc992851c7f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.438722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1314/2653] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:49.443054 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.455158 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681915c-8870-43df-9ce6-28e2a819348e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.460067 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.460813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.461005 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.461279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] PATH_INFO: `/v3/projects/a6da0052aeca4fe4beef4e09be41043e/users/68a18c3117634e929fe9f1b20091c466/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.463264 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6da0052aeca4fe4beef4e09be41043e, user_id=68a18c3117634e929fe9f1b20091c466, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.465905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.477527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1340/2654] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:49 2022] PUT /identity/v3/projects/a6da0052aeca4fe4beef4e09be41043e/users/68a18c3117634e929fe9f1b20091c466/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:49.489666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.489968 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.490236 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.518272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] MFA Rules not processed for user `68a18c3117634e929fe9f1b20091c466`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:49.519491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.593463 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.596218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.600543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1315/2655] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:49.609003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.610551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.678941 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4ff42-9b76-4ffd-bbef-9c0dd5b2d389 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.686447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.686447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.686447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.686447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.694429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a10b0-1327-438a-94b2-455ea4b8370e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.697676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.699399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318524b3-6712-44a4-9b96-a976eb003a98 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.700380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.701180 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.701533 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.701876 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] PATH_INFO: `/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.703139 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4c2033cf1c4191a4d6fca8d210d2fe)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.706104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.716077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1316/2656] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:49.761156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4c2033cf1c4191a4d6fca8d210d2fe', 'request_id': 'req-5106b153-be5f-4f40-a8e8-18432a665b7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.766929 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4c2033cf1c4191a4d6fca8d210d2fe', 'request_id': 'req-5106b153-be5f-4f40-a8e8-18432a665b7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.769451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.777118 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4c2033cf1c4191a4d6fca8d210d2fe', 'request_id': 'req-5106b153-be5f-4f40-a8e8-18432a665b7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.792393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] The token cache is being invalidate because project ba4c2033cf1c4191a4d6fca8d210d2fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:49.792393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.792393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.792393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1341/2657] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:49 2022] DELETE /identity/v3/projects/ba4c2033cf1c4191a4d6fca8d210d2fe => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:49.797459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bceac-2e7d-4908-aed1-5ac28f835aa9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.797625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bceac-2e7d-4908-aed1-5ac28f835aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.797982 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bceac-2e7d-4908-aed1-5ac28f835aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.803041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.804366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.818764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-674bceac-2e7d-4908-aed1-5ac28f835aa9 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:49.819922 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-674bceac-2e7d-4908-aed1-5ac28f835aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.824146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.868367 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106b153-be5f-4f40-a8e8-18432a665b7d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.872252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.872967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.873243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.873515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.875385 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.890572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1317/2658] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:49.899343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.899628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.899899 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.919457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:49.920702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.929255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1318/2659] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:49.933326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.937939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.940107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.942743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127f9c0-eb08-4c3f-819c-1c894e458133 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.947586 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.948211 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.948493 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.948768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.949131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1342/2660] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:49.949457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:49.949758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.951464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.963541 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1319/2661] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:32:49.968725 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.979967 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63b52-3417-40d2-a538-8f225f29389a service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.984716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.985448 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.985892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.986091 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.986682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.988672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.023436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1343/2662] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:49 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:50.027904 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.038885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec956a-9936-4110-88be-7b6172dc4e80 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.112233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.112852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.113131 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.113401 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.114134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.116053 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.137494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1320/2663] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:50.142529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.152953 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb35bb-12d2-4634-82a2-7f67f17b6b76 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.158156 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.158765 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.159048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.159329 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.160072 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.161706 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.173973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1344/2664] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:50.178188 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.187978 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe063-10cd-4b4a-947a-c4346630336b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.192672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.193262 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.193538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.193807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.194574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.196201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa926e82-c6ad-450f-9969-17e595ae966f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.206434 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1321/2665] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:50.210480 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.221252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae07e9e-1bd3-4782-9124-6b637ca49df7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.226078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.226730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.227012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.227289 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] PATH_INFO: `/v3/projects/c2e3759359904bec821802e9a94ec562/users/b8507ed19da943659320576a1bb1c61e/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.370441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3759359904bec821802e9a94ec562, user_id=b8507ed19da943659320576a1bb1c61e, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.374490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.387663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1345/2666] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:50 2022] PUT /identity/v3/projects/c2e3759359904bec821802e9a94ec562/users/b8507ed19da943659320576a1bb1c61e/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:50.399018 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.399301 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.399617 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.427007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] MFA Rules not processed for user `b8507ed19da943659320576a1bb1c61e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.428472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.485117 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:50.490725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1322/2667] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.495131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.541537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:50.556679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8976-8b26-4a0d-9aa8-a040cf9ff0dd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.561694 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.562332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.562612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.562889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.563556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.565506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.594780 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1346/2668] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:50.595914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.606396 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f195b-3202-42b2-b446-e3e740c3f872 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.661315 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.661845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.662120 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.662388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.663030 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.664971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.687153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1323/2669] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:50.691251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.701949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275158-d0a1-4908-8784-8887246f2633 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.706404 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.707039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.707315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.707585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.708370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.709947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.719981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1347/2670] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:50.723871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.733644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002333c-af73-432b-bddd-969b4584abd6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.738225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.738862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.739138 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.739414 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] PATH_INFO: `/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0/users/ac2116a07e3545d4899b0587d7fe53a7/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.754766 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f6d456139442cb973ff0dd1e8aabd0, user_id=ac2116a07e3545d4899b0587d7fe53a7, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.757469 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.771064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1324/2671] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:50 2022] PUT /identity/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0/users/ac2116a07e3545d4899b0587d7fe53a7/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:50.775117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.785222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e563-e2b2-428e-960d-ba710d30d237 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.839001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.839610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.839889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.840188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.840865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.842469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572e31d-14a5-46d4-801a-4657c5cde995 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.851865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1348/2672] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:50.856003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.866266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906a74f-8a63-4b30-a8e4-92bc787af422 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.870775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.871427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.871705 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.872004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] PATH_INFO: `/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0/users/ac2116a07e3545d4899b0587d7fe53a7/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.874599 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f6d456139442cb973ff0dd1e8aabd0, user_id=ac2116a07e3545d4899b0587d7fe53a7, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.877247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.887150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1325/2673] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:50 2022] PUT /identity/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0/users/ac2116a07e3545d4899b0587d7fe53a7/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:50.898439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.898724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.898999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.925545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] MFA Rules not processed for user `ac2116a07e3545d4899b0587d7fe53a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.926704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.992089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1349/2674] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:51.003352 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.005108 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:51.062912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37b25a-be4f-4bf3-b4e3-e5111943a10f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.066511 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.067111 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.067392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.067665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.076539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.078005 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57fcf7-0432-4e85-8dd8-901a4c1ea4a7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.093238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1326/2675] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:51.670897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.715107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:51.733222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b539aa-e4a7-4be5-a8f2-48d55af85315 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.737851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.738494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.738770 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.739044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] PATH_INFO: `/v3/users/b8507ed19da943659320576a1bb1c61e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.740238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8507ed19da943659320576a1bb1c61e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.742149 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.790309 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8507ed19da943659320576a1bb1c61e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:51.791098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8507ed19da943659320576a1bb1c61e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:51.791709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8507ed19da943659320576a1bb1c61e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:51.804735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8507ed19da943659320576a1bb1c61e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:51.812550 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8507ed19da943659320576a1bb1c61e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:51.828132 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1350/2676] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:51 2022] DELETE /identity/v3/users/b8507ed19da943659320576a1bb1c61e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:51.832809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.834339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.834677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.854137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:51.855309 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:51.901333 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:51.915016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1327/2677] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:51.924951 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.926379 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:51.982845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082dab1-461d-41cc-8cde-95fee2e7a426 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.986381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.986986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.987267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.987537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.996391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.997798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.006908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1351/2678] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:52.280826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.282508 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:52.294736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6ff5d-156f-459f-a1bf-7550eb75a2ae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.299390 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.300069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.300350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.300622 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] PATH_INFO: `/v3/projects/c2e3759359904bec821802e9a94ec562` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.301775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e3759359904bec821802e9a94ec562)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.303983 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.350893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e3759359904bec821802e9a94ec562', 'request_id': 'req-ba85e2cc-a713-4544-a677-db532f9110af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.356483 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e3759359904bec821802e9a94ec562', 'request_id': 'req-ba85e2cc-a713-4544-a677-db532f9110af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.362533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e3759359904bec821802e9a94ec562', 'request_id': 'req-ba85e2cc-a713-4544-a677-db532f9110af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.375820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] The token cache is being invalidate because project c2e3759359904bec821802e9a94ec562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:52.376139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.376761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.378277 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1328/2679] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:52 2022] DELETE /identity/v3/projects/c2e3759359904bec821802e9a94ec562 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:52.382313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.392343 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfa06-95a9-4da8-8676-2e3e9f6ca079 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.447261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.447916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.448292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.448549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] PATH_INFO: `/v3/users/ac2116a07e3545d4899b0587d7fe53a7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.449816 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2116a07e3545d4899b0587d7fe53a7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.451854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.456451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:52.501349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2116a07e3545d4899b0587d7fe53a7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.501936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2116a07e3545d4899b0587d7fe53a7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.502548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2116a07e3545d4899b0587d7fe53a7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.515150 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2116a07e3545d4899b0587d7fe53a7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.521862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d69efa0c-2536-40f9-8c3d-78143871511c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2116a07e3545d4899b0587d7fe53a7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.536103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1352/2680] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:52 2022] DELETE /identity/v3/users/ac2116a07e3545d4899b0587d7fe53a7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:52.825710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.827110 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:52.841136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2cc-a713-4544-a677-db532f9110af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.902348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.902990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.903265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.903533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] PATH_INFO: `/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.905307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f6d456139442cb973ff0dd1e8aabd0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.907189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.955921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f6d456139442cb973ff0dd1e8aabd0', 'request_id': 'req-023df63b-27ef-439c-b625-d17b8441a3bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.961536 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f6d456139442cb973ff0dd1e8aabd0', 'request_id': 'req-023df63b-27ef-439c-b625-d17b8441a3bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.968889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f6d456139442cb973ff0dd1e8aabd0', 'request_id': 'req-023df63b-27ef-439c-b625-d17b8441a3bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.981863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] The token cache is being invalidate because project f2f6d456139442cb973ff0dd1e8aabd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:52.982633 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.982790 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-023df63b-27ef-439c-b625-d17b8441a3bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:52.984361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1329/2681] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:52 2022] DELETE /identity/v3/projects/f2f6d456139442cb973ff0dd1e8aabd0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:52.989274 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.989822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.989822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.009272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:53.010615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.078227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1353/2682] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:53.086132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.086439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.086758 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.101192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:53.106458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:53.108677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.120090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1330/2683] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:53.124933 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.134875 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b6849-9569-4b36-9c8d-942b91370a6b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.140182 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.140789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.141068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.141339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.142028 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:53.142330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.143921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.155886 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1354/2684] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:53.160347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.170165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431486e-60d3-4069-b7d6-31c6f1ce7df1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.174865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.175457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.175732 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.176029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.176696 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.178817 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.204577 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1331/2685] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:53.208804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.219115 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f372-faf1-4d5e-8f7e-f2d55e5eaa60 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.277619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.278155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.278435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.279304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.279979 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.281865 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.297639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:53.303584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1355/2686] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/users => generated 281 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:53.307619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.317620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c49d-fce4-4e8e-981a-b94cc5201458 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.322052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.322650 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.322926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.323201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.323878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.325967 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.335753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1332/2687] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:53.339680 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.350043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c403ce-93a5-4996-8138-725afde87516 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.354649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.355243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.355529 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.355803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.356626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.358914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ec9aa-3f13-428e-81f8-b682867fb3f2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.369521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1356/2688] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:53.373559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.383406 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe955-a738-4a91-99bc-36681a6aa02c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.388535 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.389250 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.389549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.389838 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] PATH_INFO: `/v3/projects/a32254e4d8e44da2b8d3b25ed343a81e/users/c0b6798db61346d78b09fbecb0255b8d/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.404861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a32254e4d8e44da2b8d3b25ed343a81e, user_id=c0b6798db61346d78b09fbecb0255b8d, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.407531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.420118 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1333/2689] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:32:53 2022] PUT /identity/v3/projects/a32254e4d8e44da2b8d3b25ed343a81e/users/c0b6798db61346d78b09fbecb0255b8d/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:53.430677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.431020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.431266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.458590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] MFA Rules not processed for user `c0b6798db61346d78b09fbecb0255b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:53.459782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.524097 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1357/2690] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:53.558054 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.559708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.615860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cf74-ebca-44da-9ce0-1a668a34a591 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.619421 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.620057 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.620336 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.620612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.629377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.630777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f227e-022e-4936-b922-01847537996b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.639237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1334/2691] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:53.655639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.716879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dcee9-3a0a-48aa-a368-bb9c2672cc14 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.721443 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.722075 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.722361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.722631 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] PATH_INFO: `/v3/users/c0b6798db61346d78b09fbecb0255b8d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.723786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b6798db61346d78b09fbecb0255b8d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.725736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.770320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b6798db61346d78b09fbecb0255b8d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:53.770981 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b6798db61346d78b09fbecb0255b8d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:53.771602 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b6798db61346d78b09fbecb0255b8d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:53.783986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b6798db61346d78b09fbecb0255b8d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:53.790705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b6798db61346d78b09fbecb0255b8d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:53.798482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:53.804270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1358/2692] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:32:53 2022] DELETE /identity/v3/users/c0b6798db61346d78b09fbecb0255b8d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:53.809244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.809557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.809842 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.831296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:53.832532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.890011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1335/2693] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:53.899857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.901326 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.957538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd641d2b-6213-43b5-8bc2-99b6ddedf3b1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.961906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.962043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.962314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.962604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.972589 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.973942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.983272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1359/2694] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:54.252998 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:54.254516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:54.266537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd16bea-f03a-4aa8-ba27-756f15996c0d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:54.271271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:54.271940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:54.272248 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:54.272548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] PATH_INFO: `/v3/projects/a32254e4d8e44da2b8d3b25ed343a81e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:54.273850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a32254e4d8e44da2b8d3b25ed343a81e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:54.275877 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:54.316896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:54.324264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a32254e4d8e44da2b8d3b25ed343a81e', 'request_id': 'req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:54.330083 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a32254e4d8e44da2b8d3b25ed343a81e', 'request_id': 'req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:54.336228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a32254e4d8e44da2b8d3b25ed343a81e', 'request_id': 'req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:54.348846 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] The token cache is being invalidate because project a32254e4d8e44da2b8d3b25ed343a81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:54.349132 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:54.349743 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73ae7ed1-11e8-4870-a31a-7acd034c9a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:54.351223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1336/2695] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:32:54 2022] DELETE /identity/v3/projects/a32254e4d8e44da2b8d3b25ed343a81e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:03.979127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=27 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:04.345875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:06.582522 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.583947 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.661610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cbad-df30-4eea-8992-6bfedf8409a1 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.717812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.718456 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.718738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.719016 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] PATH_INFO: `/v3/users/de6d383db02e46de92dea29671417a51` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.720217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6d383db02e46de92dea29671417a51)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.722161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.766221 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6d383db02e46de92dea29671417a51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:06.766855 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6d383db02e46de92dea29671417a51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:06.767447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6d383db02e46de92dea29671417a51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:06.779822 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6d383db02e46de92dea29671417a51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:06.786636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6d383db02e46de92dea29671417a51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:06.798435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1360/2696] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:33:06 2022] DELETE /identity/v3/users/de6d383db02e46de92dea29671417a51 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:06.802874 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.803656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.803775 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.832677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:06.833824 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.890433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1337/2697] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:06.900206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.901678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.922545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:06.975919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bab759-a3c6-49cd-a6b6-e1ddf8812228 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.979505 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.980136 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.980411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.980678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.989481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.990889 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8ae6e-1918-40aa-a383-f92b19c8d6b1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.000491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1361/2698] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:07.265013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.266474 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.278640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc66aad-eab7-4a93-b14a-121a49358bf8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.295803 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.296657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.296946 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.297221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] PATH_INFO: `/v3/projects/e81fc1a0c6fe42e490d276d19d8cc1c0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.298572 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81fc1a0c6fe42e490d276d19d8cc1c0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.300589 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.358892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81fc1a0c6fe42e490d276d19d8cc1c0', 'request_id': 'req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:07.364797 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81fc1a0c6fe42e490d276d19d8cc1c0', 'request_id': 'req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:07.373346 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81fc1a0c6fe42e490d276d19d8cc1c0', 'request_id': 'req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:07.390545 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] The token cache is being invalidate because project e81fc1a0c6fe42e490d276d19d8cc1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:07.390863 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:07.391448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a3a2570-3c54-44cb-b180-ea93aa4a38f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:07.392979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1338/2699] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:33:07 2022] DELETE /identity/v3/projects/e81fc1a0c6fe42e490d276d19d8cc1c0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:07.403126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.403126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.403126 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.449800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:07.451554 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.509521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1362/2700] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:07.517935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.518218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.518490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.537688 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:07.538855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.547531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1339/2701] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:07.551812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.564065 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c845-164f-416d-bf17-bc054ab3779d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.569376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.569996 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.570307 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.570586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.571778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:07.571778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.573184 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.585585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1363/2702] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:07.590034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.599894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c48cb-b5af-417b-adcf-545c285f86f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.604753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.605350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.605680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.605947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.607095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.609025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.643498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1340/2703] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:07.647622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.669620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f90b9-3ea0-41ff-b997-12106094c9d0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.746677 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.747320 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.747627 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.747898 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.748703 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.750746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.766510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:07.773495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1364/2704] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:07.777635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.788997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b709c8-48f3-417d-8525-ee097c2dcd5c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.794042 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.794628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.794909 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.795184 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.795901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.797515 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.807616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1341/2705] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:07.811605 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.821437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c980bd-86c3-4c20-a109-daa228a54e6f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.826501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.827099 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.827380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.827654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.828378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.829959 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ae583-53ef-4fec-b5e8-cef87fd82a7b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.842532 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1365/2706] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:07.846637 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.856613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cf1e-60e9-497c-a368-db6acdc93cc7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.861104 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.861748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.862026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.862303 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] PATH_INFO: `/v3/projects/0d6c21f221344a35938d748d4eebce66/users/6699ab7225304e1da833bd9fea31f496/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.876481 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6c21f221344a35938d748d4eebce66, user_id=6699ab7225304e1da833bd9fea31f496, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.879129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.892139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1342/2707] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:07 2022] PUT /identity/v3/projects/0d6c21f221344a35938d748d4eebce66/users/6699ab7225304e1da833bd9fea31f496/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:07.902511 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.902803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.903079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.930583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] MFA Rules not processed for user `6699ab7225304e1da833bd9fea31f496`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:07.931811 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.996043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1366/2708] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:08.007860 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.009630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.024224 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:08.066745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae9b-0d1b-4bae-b1dd-5f86e0e40033 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.070384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.070989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.071269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.071540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.080455 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.082074 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f33ee-09d0-4e96-8684-760037f7885f service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.090826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1343/2709] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:08.114808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.174941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb85e0-8868-447d-9433-35976c49f056 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.179846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.180496 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.180773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.181039 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] PATH_INFO: `/v3/users/6699ab7225304e1da833bd9fea31f496` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.182176 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6699ab7225304e1da833bd9fea31f496)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.184092 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.227147 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6699ab7225304e1da833bd9fea31f496'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.227793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6699ab7225304e1da833bd9fea31f496'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.228438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6699ab7225304e1da833bd9fea31f496'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.240564 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6699ab7225304e1da833bd9fea31f496'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.248058 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6699ab7225304e1da833bd9fea31f496'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.251610 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:08.262753 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1367/2710] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:33:08 2022] DELETE /identity/v3/users/6699ab7225304e1da833bd9fea31f496 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:08.267704 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.268026 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.268306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.287344 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:08.288581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.345940 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1344/2711] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:08.355648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.357131 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.416509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cd88-d5ce-48ee-b7fc-27252c166b60 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.420112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.420718 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.420991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.421260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.430178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.431584 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7cd3c-4742-4368-ac51-000f4118028a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.440768 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1368/2712] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:08.705734 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.707341 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.721283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e75ca-fa46-481d-8cb1-4e91ff87b594 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.726760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.727605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.727896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.728202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] PATH_INFO: `/v3/projects/0d6c21f221344a35938d748d4eebce66` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.729365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6c21f221344a35938d748d4eebce66)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.731264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.779947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6c21f221344a35938d748d4eebce66', 'request_id': 'req-7eb41456-855a-4c68-a7a2-5d94bd466bdc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.786080 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6c21f221344a35938d748d4eebce66', 'request_id': 'req-7eb41456-855a-4c68-a7a2-5d94bd466bdc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.794404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6c21f221344a35938d748d4eebce66', 'request_id': 'req-7eb41456-855a-4c68-a7a2-5d94bd466bdc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.807581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] The token cache is being invalidate because project 0d6c21f221344a35938d748d4eebce66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:08.807879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.808529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7eb41456-855a-4c68-a7a2-5d94bd466bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:08.810745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1345/2713] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:33:08 2022] DELETE /identity/v3/projects/0d6c21f221344a35938d748d4eebce66 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:08.815501 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.815946 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.816268 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.836245 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:08.837451 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.898762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1369/2714] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:08.908231 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.908584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.908865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.929284 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:08.930491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:08.938993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1346/2715] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:08.943336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.953432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10da634-cc59-4515-a072-9650ed4f2681 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.958032 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.958622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.958896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.959162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.959842 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:08.960161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.961736 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.972986 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1370/2716] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:09.032557 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.042100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735370a-9e8a-4987-90a4-e7978431eede None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.047348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.047976 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.048269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.048540 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.049212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.051092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.082258 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1347/2717] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:09 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:09.086397 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.096521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870a371-c83f-4944-8988-23b727263788 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.150258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.150789 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.151066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.151339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.152024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.153934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.175577 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1371/2718] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:09 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:09.180172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.191023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0392c1a-3c41-47bf-8e5e-b60378b152d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.195773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.196409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.196689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.196962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.197644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.199223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.204366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:09.209722 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1348/2719] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:09.213678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.223477 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16532b6a-1f01-4a97-afa4-5bf2f070f379 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.228543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.229142 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.229460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.229757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.230435 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.232518 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9b8dc-ea0c-4cd1-88fc-a67b5e7eb028 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.242934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1372/2720] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:09.247225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.258004 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d590a7-1e8f-41bb-9e29-9b3414bba710 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.262795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.263458 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.263739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.264069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] PATH_INFO: `/v3/projects/0ae88b292f5c46a2883b75c18cfe94bc/users/4b1c1c5a726942f885b690b73050abc4/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.279472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae88b292f5c46a2883b75c18cfe94bc, user_id=4b1c1c5a726942f885b690b73050abc4, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.282172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.295357 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1349/2721] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:09 2022] PUT /identity/v3/projects/0ae88b292f5c46a2883b75c18cfe94bc/users/4b1c1c5a726942f885b690b73050abc4/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:09.305722 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.306002 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.306270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.314724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:09.333964 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] MFA Rules not processed for user `4b1c1c5a726942f885b690b73050abc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:09.335153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:09.416805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1373/2722] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:09.427912 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.429836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:09.485350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796640b6-7e93-4f6d-8e97-3ccc821f1743 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.488972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.489582 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.489862 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.490137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.498384 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.499805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.508580 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1350/2723] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:13.415600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:13.417040 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:13.510672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7987b0-7877-4f84-9e34-7849bb1d9e08 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:13.517369 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:13.517987 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.518263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.518536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.520349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:13.532022 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:13.533583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:13.543455 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1374/2724] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:19.504731 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:21.793852 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:21.803153 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8fba-c188-4132-bed3-97859db3abeb service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:21.807323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:21.807925 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:21.808233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:21.808504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:21.810869 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:21.867812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:21.869268 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400475ee-0d68-4eba-b440-1bb584cbf1d3 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:21.878873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1351/2725] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:22.517809 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:22.578113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f87d08-2234-4cfe-bb17-8def029db0d2 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:22.582658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:22.583282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:22.583561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:22.583832 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] PATH_INFO: `/v3/users/4b1c1c5a726942f885b690b73050abc4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:22.585011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1c1c5a726942f885b690b73050abc4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:22.586919 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:22.592629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:22.638654 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1c1c5a726942f885b690b73050abc4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:22.639283 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1c1c5a726942f885b690b73050abc4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:22.639941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1c1c5a726942f885b690b73050abc4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:22.652364 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1c1c5a726942f885b690b73050abc4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:22.659827 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1c1c5a726942f885b690b73050abc4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:22.674757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1375/2726] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:33:22 2022] DELETE /identity/v3/users/4b1c1c5a726942f885b690b73050abc4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:22.683161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:22.683161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:22.683161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:22.699839 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:22.701555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:22.764381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1352/2727] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:22.774882 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:22.776416 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:22.840324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f124be-ea7f-4f59-a91f-757e03260c5b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:22.844684 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:22.845370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:22.845854 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:22.846322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:22.866869 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:22.868444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7196e-dd16-4ef6-8a96-e46802109ba7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:22.880549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1376/2728] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:23.157613 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.159068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:23.171121 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84470-2b46-4cb7-91cd-30e4ff642b50 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.175804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.176478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.176763 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.177063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] PATH_INFO: `/v3/projects/0ae88b292f5c46a2883b75c18cfe94bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.178226 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae88b292f5c46a2883b75c18cfe94bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.180630 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.236906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae88b292f5c46a2883b75c18cfe94bc', 'request_id': 'req-2510d174-babc-4ffe-8670-008838f8ce2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:23.243003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae88b292f5c46a2883b75c18cfe94bc', 'request_id': 'req-2510d174-babc-4ffe-8670-008838f8ce2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:23.250884 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae88b292f5c46a2883b75c18cfe94bc', 'request_id': 'req-2510d174-babc-4ffe-8670-008838f8ce2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:23.262457 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] The token cache is being invalidate because project 0ae88b292f5c46a2883b75c18cfe94bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:23.262777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:23.263916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2510d174-babc-4ffe-8670-008838f8ce2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:23.265479 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1353/2729] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:33:23 2022] DELETE /identity/v3/projects/0ae88b292f5c46a2883b75c18cfe94bc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:23.270603 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.270892 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.271164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.290110 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:23.291222 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:23.348790 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1377/2730] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:23.356989 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.357520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.357520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.377212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:23.378467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:23.386435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1354/2731] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:23.392089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.401678 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da82845-9d8c-46e6-a439-4b47d0c87ed8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.406238 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.406846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.407128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.407419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.408125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:23.408430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.410014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.421612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1378/2732] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:23.425861 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.435764 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1354d-a1a9-44c0-a092-64f10a32382d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.440624 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.441274 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.441552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.441847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.442510 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.444709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.458919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:23.472472 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1355/2733] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:23.476731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.487210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589941f-8c0d-4ece-ba63-db04c11b91f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.495813 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:23.554559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.555263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.555380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.555615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.556295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.558191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.579128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1379/2734] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:23 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:23.583246 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.593422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4083b-039b-47e2-b3ac-63c98b662881 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.598253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.598855 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.599137 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.599413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.600128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.601713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.612334 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1356/2735] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:23.616224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.626643 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f965af0-fc96-4414-9601-06ae7385b3a5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.631168 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.631762 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.632069 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.632345 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.633030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.634598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc8104-86e0-439c-b80f-c83ccee3f43e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.644862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1380/2736] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:23.648718 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.659007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f03ad-0fff-49a2-a523-7eda407c535c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.663568 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.664237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.664521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.664795 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] PATH_INFO: `/v3/projects/f63d8ac72593445089042fd7c3a31651/users/edd0f098e40c4c2da8d55df9a8fa0912/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.679699 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63d8ac72593445089042fd7c3a31651, user_id=edd0f098e40c4c2da8d55df9a8fa0912, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.682379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.695323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1357/2737] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:23 2022] PUT /identity/v3/projects/f63d8ac72593445089042fd7c3a31651/users/edd0f098e40c4c2da8d55df9a8fa0912/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:23.700425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9938-9072-47cb-bea6-7568652a134d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.700738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9938-9072-47cb-bea6-7568652a134d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.701012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9938-9072-47cb-bea6-7568652a134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.719910 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-347f9938-9072-47cb-bea6-7568652a134d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:23.721304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-347f9938-9072-47cb-bea6-7568652a134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:23.777778 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1381/2738] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:23.787777 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:23.789432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:23.866091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d12bd9-5ed5-4b4c-8541-1c78e4b74304 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:23.869646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:23.870244 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:23.870523 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:23.870792 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:23.879525 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:23.880960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:23.890176 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1358/2739] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:28.278831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.279361 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.279521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.308639 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] MFA Rules not processed for user `edd0f098e40c4c2da8d55df9a8fa0912`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:28.310392 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:28.389664 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1382/2740] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:28.394168 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.406760 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dccbb-4b2f-470b-83e5-84458af875a7 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.411668 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649d77-7513-4123-9207-7398100de963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.412332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649d77-7513-4123-9207-7398100de963 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.412649 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649d77-7513-4123-9207-7398100de963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.412964 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649d77-7513-4123-9207-7398100de963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.413798 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96649d77-7513-4123-9207-7398100de963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.415843 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96649d77-7513-4123-9207-7398100de963 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.448916 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1359/2741] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:28 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:28.453681 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.464895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3f03c-debc-4fbe-a1a9-54b3035bd4aa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.483316 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:28.525255 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.525879 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.526164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.526442 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.527107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.529178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.552331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1383/2742] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:28 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:28.556598 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649d77-7513-4123-9207-7398100de963 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.568388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649d77-7513-4123-9207-7398100de963 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.573426 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.574124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.574420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.574693 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.575397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.577191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.588664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1360/2743] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:28.592558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.602699 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef29f6-f6a8-4f66-9181-e4a3a2a2b51b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.607901 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.608612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.608909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.609160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] PATH_INFO: `/v3/projects/a1cdd618739b4319bbaf7a3258f6937f/users/81504d3f2d9741bea68401206237c274/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.625843 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cdd618739b4319bbaf7a3258f6937f, user_id=81504d3f2d9741bea68401206237c274, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.628629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.642632 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1384/2744] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:28 2022] PUT /identity/v3/projects/a1cdd618739b4319bbaf7a3258f6937f/users/81504d3f2d9741bea68401206237c274/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:28.646517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.656639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab272-f6e9-48a9-89ca-fc46e5a5ef6c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.715235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.715905 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.716269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.716488 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.717187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.718826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922d4fe-75d3-4bf4-a7ff-43ec3c80987d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.730090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1361/2745] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:28.734900 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:28.745506 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a8cf-91d1-4e58-b2ac-3eebb3546192 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:28.750715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:28.751378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:28.751655 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:28.751924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] PATH_INFO: `/v3/projects/a1cdd618739b4319bbaf7a3258f6937f/users/81504d3f2d9741bea68401206237c274/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.754027 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cdd618739b4319bbaf7a3258f6937f, user_id=81504d3f2d9741bea68401206237c274, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.756685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.766895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1385/2746] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:28 2022] PUT /identity/v3/projects/a1cdd618739b4319bbaf7a3258f6937f/users/81504d3f2d9741bea68401206237c274/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:32.309113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b338-4e56-4c81-b918-3ae308926e49 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:32.310628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b338-4e56-4c81-b918-3ae308926e49 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:32.311267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b338-4e56-4c81-b918-3ae308926e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:32.320373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:32.350142 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c507b338-4e56-4c81-b918-3ae308926e49 None None] MFA Rules not processed for user `fd829e78e710440687678701fea3cf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:32.351530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c507b338-4e56-4c81-b918-3ae308926e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:32.493273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1362/2747] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:33.569161 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:33.572031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:33.693560 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a62c2e-efe3-46a0-a7c6-74bfc17b74b7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:33.700726 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:33.702001 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:33.704425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:33.705079 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] PATH_INFO: `/v3/users/a79b35b536b7442b8e6e2b19583c40a9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:33.710779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79b35b536b7442b8e6e2b19583c40a9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:33.712191 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:33.795779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79b35b536b7442b8e6e2b19583c40a9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:33.796474 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79b35b536b7442b8e6e2b19583c40a9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:33.797089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79b35b536b7442b8e6e2b19583c40a9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:33.812051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79b35b536b7442b8e6e2b19583c40a9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:33.820563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79b35b536b7442b8e6e2b19583c40a9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:33.825308 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:33.850279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1386/2748] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:33:33 2022] DELETE /identity/v3/users/a79b35b536b7442b8e6e2b19583c40a9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:33.852654 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459f34-5132-42e9-bdce-27526f36fdaf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:33.853438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459f34-5132-42e9-bdce-27526f36fdaf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:33.853936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459f34-5132-42e9-bdce-27526f36fdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:33.888550 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-12459f34-5132-42e9-bdce-27526f36fdaf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:33.890329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-12459f34-5132-42e9-bdce-27526f36fdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.053820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1363/2749] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:34.060921 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:34.062574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.139504 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.140266 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.140635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.182563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] MFA Rules not processed for user `81504d3f2d9741bea68401206237c274`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:34.183910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.224490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a01340-c550-48f4-b175-bf1b0bf74785 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:34.228805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:34.229880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.229880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.230286 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.255857 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:34.257731 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:34.272673 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1387/2750] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:33:34.307498 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1364/2751] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:34.320417 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:34.322120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.402202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f139b-bb36-4df6-b470-f4e0a96cfed2 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:34.407757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f0130283f8f42bb94a894b5a1b55d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:34.408376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.408653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.409597 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.411772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.491468 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:34.492866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9192513-27b9-4e62-b341-a0e5bdbd679a service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:34.505621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1388/2752] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:34.628311 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:34.630739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.661906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.663165 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094b310-cc9b-4930-89f0-3a11fd2de7e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:34.663662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.664276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.680443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:34.683524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.684213 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.684716 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] PATH_INFO: `/v3/projects/aacf5efd0b274d9484e0f08416404ff6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.686265 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacf5efd0b274d9484e0f08416404ff6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:34.688490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:34.704544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:34.705903 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.739387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:34.759453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacf5efd0b274d9484e0f08416404ff6', 'request_id': 'req-489f273c-2126-4d7a-98c6-24fe2f2e7097'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.767734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:34.768093 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacf5efd0b274d9484e0f08416404ff6', 'request_id': 'req-489f273c-2126-4d7a-98c6-24fe2f2e7097'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.775501 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacf5efd0b274d9484e0f08416404ff6', 'request_id': 'req-489f273c-2126-4d7a-98c6-24fe2f2e7097'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.791438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] The token cache is being invalidate because project aacf5efd0b274d9484e0f08416404ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:34.791815 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.793393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.794020 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1389/2753] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:34.795784 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1365/2754] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:33:34 2022] DELETE /identity/v3/projects/aacf5efd0b274d9484e0f08416404ff6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:34.806745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:34.808400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.851281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:34.852943 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:34.885614 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f273c-2126-4d7a-98c6-24fe2f2e7097 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:34.890787 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:34.891429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.891708 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.892119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] PATH_INFO: `/v3/users/68a18c3117634e929fe9f1b20091c466` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.893307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a18c3117634e929fe9f1b20091c466)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:34.895225 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:34.929995 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f50bef-19f9-4cb3-8e86-0fc036a1efbc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:34.948781 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a18c3117634e929fe9f1b20091c466'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.949526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a18c3117634e929fe9f1b20091c466'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.951135 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a18c3117634e929fe9f1b20091c466'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.963562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a18c3117634e929fe9f1b20091c466'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.970832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a18c3117634e929fe9f1b20091c466'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:34.986101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1366/2755] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:33:34 2022] DELETE /identity/v3/users/68a18c3117634e929fe9f1b20091c466 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:34.992106 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:34.992729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:34.993011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:34.993282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:34.995041 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:35.058206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:35.060604 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9f135-e4c6-4ecc-b7d2-0863a5b66623 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:35.070117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1390/2756] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 10:33:35.343127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:35.344660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:35.439802 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:35.472179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25827-a466-48aa-8070-0bfbfb545429 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:35.478006 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:35.479001 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.479517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.480013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] PATH_INFO: `/v3/projects/a6da0052aeca4fe4beef4e09be41043e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.481538 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6da0052aeca4fe4beef4e09be41043e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:35.486517 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:35.541530 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6da0052aeca4fe4beef4e09be41043e', 'request_id': 'req-67260472-6cb3-4db1-ab0b-56a9b744c0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:35.547562 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6da0052aeca4fe4beef4e09be41043e', 'request_id': 'req-67260472-6cb3-4db1-ab0b-56a9b744c0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:35.554505 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6da0052aeca4fe4beef4e09be41043e', 'request_id': 'req-67260472-6cb3-4db1-ab0b-56a9b744c0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:35.568615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] The token cache is being invalidate because project a6da0052aeca4fe4beef4e09be41043e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:35.568947 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:35.569528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67260472-6cb3-4db1-ab0b-56a9b744c0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:35.575726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1367/2757] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:33:35 2022] DELETE /identity/v3/projects/a6da0052aeca4fe4beef4e09be41043e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:35.577438 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.577745 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.578024 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.609948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:35.611662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:35.633294 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:35.684916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1391/2758] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:35.690400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.690782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.691136 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.718358 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:35.719690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:35.732762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1368/2759] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:35.736621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:35.755580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f504bf-7303-49fb-bdb0-90977fcd95e7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:35.763591 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:35.764270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.764534 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.764804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.765508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:35.767096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:35.769060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:35.791687 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1392/2760] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:33:35.794800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:35.805192 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc4a28-0d3a-42cb-b8dc-dbe3a1b4b3c3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:35.813257 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:35.813895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.814175 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.814453 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.815355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:35.818651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:35.864240 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1369/2761] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:35 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:35.867893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:35.881891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c18e01-87e5-46a5-ad57-b0a8659d9b7b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:35.951835 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:35.952490 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:35.952774 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:35.953047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:35.953715 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:35.956234 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:35.988074 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1393/2762] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:35 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:35.991993 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.006326 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17dd67-ab53-41f2-bdff-759c780d83e7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.017101 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.017730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.018366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.018366 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.020203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.021253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.033149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1370/2763] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:36.039060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.047991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6c45-ea6d-435e-8552-03dadccdedc7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.052784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.053407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.053948 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.054224 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.054924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.056574 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd324b4-f48c-4517-b82d-97716b885f96 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.077120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1394/2764] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:36.078182 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.094717 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907567-78db-431a-84b7-05bb3d2bfe4f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.100420 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.101023 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.101404 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.101774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] PATH_INFO: `/v3/projects/b067e73d40874e6caabafeee589da2f7/users/f08642dd5cfe4ce99e3e1ddc180bc782/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.123873 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b067e73d40874e6caabafeee589da2f7, user_id=f08642dd5cfe4ce99e3e1ddc180bc782, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.126845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.146547 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1371/2765] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:36 2022] PUT /identity/v3/projects/b067e73d40874e6caabafeee589da2f7/users/f08642dd5cfe4ce99e3e1ddc180bc782/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:36.158914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.159202 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.159486 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.194895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] MFA Rules not processed for user `f08642dd5cfe4ce99e3e1ddc180bc782`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:36.196153 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:36.304336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1395/2766] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:36.311871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.449238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9f037-8bd0-4a62-b877-a1825e93a1a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.459552 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.460330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.460678 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.461526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.462368 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.465397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.481782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:36.514746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.517385 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1372/2767] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:33:36 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:36.531107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea842d1e-e51d-412a-98ea-395ae3969334 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.550863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:36.611771 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.612657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.612945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.613217 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.617398 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.619563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.649945 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1396/2768] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:33:36 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:36.653710 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.665473 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3579-9e0a-494f-99ae-cfda53ba80f4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.671526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.672267 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.672546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.672936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.673878 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.675885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.688859 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1373/2769] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:36.693188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.705052 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa2d43-6f70-4cdd-9e64-425647902125 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.712048 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.713185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.713598 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.713890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] PATH_INFO: `/v3/projects/e800d160e3a4403a9576fc58f33c6b79/users/aa0ea7b797284454a41248f5ef7ba901/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.737587 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e800d160e3a4403a9576fc58f33c6b79, user_id=aa0ea7b797284454a41248f5ef7ba901, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.740662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.760977 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1397/2770] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:36 2022] PUT /identity/v3/projects/e800d160e3a4403a9576fc58f33c6b79/users/aa0ea7b797284454a41248f5ef7ba901/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:36.765149 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.776682 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff752-f580-4fab-a079-8473704ff4b4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.863747 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.864712 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.864999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.865273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.865971 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.867626 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fdebb-3e15-4bf3-969b-372d9a043b66 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.884124 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1374/2771] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:36.885330 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:36.898716 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507518-6b70-41fb-b7b8-f23a4290901a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:36.914817 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:36.915521 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.915807 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.916127 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] PATH_INFO: `/v3/projects/e800d160e3a4403a9576fc58f33c6b79/users/aa0ea7b797284454a41248f5ef7ba901/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:36.918424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e800d160e3a4403a9576fc58f33c6b79, user_id=aa0ea7b797284454a41248f5ef7ba901, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:36.921717 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:36.942199 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1398/2772] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:33:36 2022] PUT /identity/v3/projects/e800d160e3a4403a9576fc58f33c6b79/users/aa0ea7b797284454a41248f5ef7ba901/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:36.953215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:36.953868 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:36.954412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:37.000484 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] MFA Rules not processed for user `aa0ea7b797284454a41248f5ef7ba901`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:37.002621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:37.105960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1375/2773] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:37.119538 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:37.122662 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:37.234112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4341-8377-41a0-a4b7-3a0a05cd4185 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:37.246491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:37.246491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:37.246491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:37.246491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:37.319466 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:37.321386 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffdece-12b2-4249-9a07-38b7d7c57924 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:37.336561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1399/2774] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:43.342035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.357621 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e96dab-f4c5-4f6a-a01b-c7061299e876 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.368939 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.369849 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.370464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.371009 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.384283 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.388492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e3696-c656-4488-bbc0-27cf7197f974 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.403753 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1376/2775] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:44.962418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.962908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.963029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.976580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:44.983315 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] MFA Rules not processed for user `edd0f098e40c4c2da8d55df9a8fa0912`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:44.984541 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.045625 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1400/2776] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:47.874724 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.875228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.875422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.906924 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:47.911338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:47.959820 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:47.976711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1377/2777] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:47.990991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.992531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:48.073619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2049e5-d59e-4db1-83fe-ffeb1348f017 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.079723 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.082258 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.082653 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.083031 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.093657 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.101775 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab0d98-bc40-4f8c-9438-ee70f9cb5d95 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.120188 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1401/2778] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:33:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:33:57.970350 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:58.114588 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=54 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:08.758314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:08.820161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d577eb-1655-4693-961d-2be4701454f6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:08.824555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:08.825178 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:08.825454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:08.825725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] PATH_INFO: `/v3/users/f08642dd5cfe4ce99e3e1ddc180bc782` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:08.826875 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08642dd5cfe4ce99e3e1ddc180bc782)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:08.828825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:08.874286 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08642dd5cfe4ce99e3e1ddc180bc782'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:08.874945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08642dd5cfe4ce99e3e1ddc180bc782'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:08.875558 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08642dd5cfe4ce99e3e1ddc180bc782'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:08.897264 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08642dd5cfe4ce99e3e1ddc180bc782'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:08.915796 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08642dd5cfe4ce99e3e1ddc180bc782'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:08.929744 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1378/2779] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:08 2022] DELETE /identity/v3/users/f08642dd5cfe4ce99e3e1ddc180bc782 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:08.934175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:08.934679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:08.934793 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:08.954338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:08.955616 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:09.016277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1402/2780] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:09.027539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:09.028987 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:09.086361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541f69d-ba1d-42f9-839b-097cc3bad050 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:09.090516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:09.091133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:09.091410 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:09.091681 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:09.102381 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:09.103907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:09.116779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1379/2781] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:09.385261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:09.386666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:09.409213 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adb93-8868-42a6-af55-af8a06f4e653 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:09.415862 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:09.416647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:09.416951 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:09.417243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] PATH_INFO: `/v3/projects/b067e73d40874e6caabafeee589da2f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:09.418527 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b067e73d40874e6caabafeee589da2f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:09.420563 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:09.470836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b067e73d40874e6caabafeee589da2f7', 'request_id': 'req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.477366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b067e73d40874e6caabafeee589da2f7', 'request_id': 'req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.484912 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b067e73d40874e6caabafeee589da2f7', 'request_id': 'req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.497049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] The token cache is being invalidate because project b067e73d40874e6caabafeee589da2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:09.497337 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.497973 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.499459 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1403/2782] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:09 2022] DELETE /identity/v3/projects/b067e73d40874e6caabafeee589da2f7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:09.503665 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:09.513343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291feddf-9161-4fd3-a40b-029bff4ccaf4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:09.533145 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:09.575527 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:09.576198 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:09.576478 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:09.576751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] PATH_INFO: `/v3/users/aa0ea7b797284454a41248f5ef7ba901` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:09.577911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0ea7b797284454a41248f5ef7ba901)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:09.579974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:09.623578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0ea7b797284454a41248f5ef7ba901'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.624491 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0ea7b797284454a41248f5ef7ba901'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.625129 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0ea7b797284454a41248f5ef7ba901'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.637229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0ea7b797284454a41248f5ef7ba901'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.644041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4df3b26-934a-40c9-8ee3-b27bdb611d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0ea7b797284454a41248f5ef7ba901'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:09.656667 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1380/2783] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:09 2022] DELETE /identity/v3/users/aa0ea7b797284454a41248f5ef7ba901 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:09.937844 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:09.939497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:09.953003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbac4-142d-4d29-bd6f-fd55a78b8ce6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.014301 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.014939 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.015218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.015489 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] PATH_INFO: `/v3/projects/e800d160e3a4403a9576fc58f33c6b79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.017181 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e800d160e3a4403a9576fc58f33c6b79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.020011 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.051828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:10.070030 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e800d160e3a4403a9576fc58f33c6b79', 'request_id': 'req-ff2fbb67-2786-44d3-845c-46f8b30dcfae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.075421 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e800d160e3a4403a9576fc58f33c6b79', 'request_id': 'req-ff2fbb67-2786-44d3-845c-46f8b30dcfae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.082893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e800d160e3a4403a9576fc58f33c6b79', 'request_id': 'req-ff2fbb67-2786-44d3-845c-46f8b30dcfae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.093834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] The token cache is being invalidate because project e800d160e3a4403a9576fc58f33c6b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:10.094124 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.094772 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff2fbb67-2786-44d3-845c-46f8b30dcfae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.096324 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1404/2784] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:09 2022] DELETE /identity/v3/projects/e800d160e3a4403a9576fc58f33c6b79 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:10.101252 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.101759 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.101870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.122229 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:10.123422 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.184652 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1381/2785] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:10.194383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.194668 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.194944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.215210 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:10.216526 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.224895 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1405/2786] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:10.229134 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.243417 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c72f9-3ce7-4d22-8cb5-5790b5284f28 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.249113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.250259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.250259 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.250467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.251531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:10.253113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.254073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.266041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1382/2787] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:10.271128 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.280884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc7b2f-0c7e-4785-b9c6-b7c4db851801 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.285805 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.286514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.286866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.287266 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.288070 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.290084 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.322649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1406/2788] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:10.323917 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.334551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca55fa-73ee-4a11-a47a-6e917a953e80 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.359238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:10.390561 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.391172 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.392114 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.392395 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.393087 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.394984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.415397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1383/2789] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:10.419494 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.430444 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9679a-4543-4bdb-b831-bd0fd1171f6d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.435015 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.435615 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.435899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.436200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.436881 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.439436 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.450606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1407/2790] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:10.454657 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.464646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3c127-eb96-4a70-9cb3-445ab3b0e9ba admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.469394 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.470063 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.470387 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.470706 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.471499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.473130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca808a-d4f5-4739-9960-0814967353ff admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.484529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1384/2791] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:10.489090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.499122 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8374-51d1-45f4-9c84-9a6651a861ae admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.504424 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.505076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.505362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.505636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] PATH_INFO: `/v3/projects/b453e60ce7294518811621a8571829b1/users/8da2919ff00e4e6d88201c7667e7f37f/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.520911 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b453e60ce7294518811621a8571829b1, user_id=8da2919ff00e4e6d88201c7667e7f37f, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.523557 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.538003 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1408/2792] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:10 2022] PUT /identity/v3/projects/b453e60ce7294518811621a8571829b1/users/8da2919ff00e4e6d88201c7667e7f37f/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:10.549574 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.549857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.550133 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.579516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] MFA Rules not processed for user `8da2919ff00e4e6d88201c7667e7f37f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:10.580673 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.648446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1385/2793] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:10.659355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.661089 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.719125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a1be-81a5-4221-abbe-8a77a3b9cf43 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.723510 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.724160 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.724449 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.724730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.733727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.735940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ddfe0-ff02-4320-aafd-9d8e2c5287e8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.745197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1409/2794] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:10.766610 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.828805 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea8c18-a8ff-4384-8b8e-6933d020d81b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.833459 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.834031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.834314 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.834590 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] PATH_INFO: `/v3/users/8da2919ff00e4e6d88201c7667e7f37f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.835757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da2919ff00e4e6d88201c7667e7f37f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.837727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.877516 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:10.884386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da2919ff00e4e6d88201c7667e7f37f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.885269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da2919ff00e4e6d88201c7667e7f37f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.886075 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da2919ff00e4e6d88201c7667e7f37f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.897923 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da2919ff00e4e6d88201c7667e7f37f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.904616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da2919ff00e4e6d88201c7667e7f37f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:10.917887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1386/2795] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:10 2022] DELETE /identity/v3/users/8da2919ff00e4e6d88201c7667e7f37f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:10.922484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.923384 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.923672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.943263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:10.944600 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.993897 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:11.006806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1410/2796] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:11.016539 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.017969 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.075419 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b4e77-f4f9-421e-bc6f-9c4d7d3fb3aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.079032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.079642 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.079921 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.080221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.089170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.090526 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a31e-6887-4c1c-959f-c5cfcc0ac545 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.100348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1387/2797] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:11.637291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.638828 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.651695 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418f99-c77b-45bb-96ae-a7630a2c4584 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.656447 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.657125 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.657405 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.657679 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] PATH_INFO: `/v3/projects/b453e60ce7294518811621a8571829b1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.658831 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b453e60ce7294518811621a8571829b1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.660782 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.708419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b453e60ce7294518811621a8571829b1', 'request_id': 'req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.713760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b453e60ce7294518811621a8571829b1', 'request_id': 'req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.720801 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b453e60ce7294518811621a8571829b1', 'request_id': 'req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.731734 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] The token cache is being invalidate because project b453e60ce7294518811621a8571829b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:11.732076 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.732704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bc98fef2-d3bf-4f9b-b9b2-4b5c75282bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.734216 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1411/2798] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:11 2022] DELETE /identity/v3/projects/b453e60ce7294518811621a8571829b1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:11.739116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.739584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.739702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.759445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:11.760658 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.826191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1388/2799] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:11.834896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897679d-4aee-425d-8548-25885b509697 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.835185 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897679d-4aee-425d-8548-25885b509697 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.835458 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897679d-4aee-425d-8548-25885b509697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.855262 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8897679d-4aee-425d-8548-25885b509697 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:11.856756 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8897679d-4aee-425d-8548-25885b509697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.865170 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1412/2800] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:11.869429 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.880289 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb7262-c712-4ac7-9eaf-1ad19a200976 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.886462 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.887099 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.887379 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.887656 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.888392 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:11.889062 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.890292 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.909543 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1389/2801] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:11.910135 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897679d-4aee-425d-8548-25885b509697 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.923259 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897679d-4aee-425d-8548-25885b509697 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.930419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.933112 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.933429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.934880 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.935580 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.937856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.981704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1413/2802] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:34:11 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:11.986551 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.991737 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.993470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.998528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968cfaf-4ffc-479b-924e-9a898f325d9a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.058162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:12.076245 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.076879 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.077170 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.077444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.078144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.080127 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.105936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1390/2803] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:34:11 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:12.107325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.118389 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0657202-5b39-45dc-84dd-613242a16ac1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.121102 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aae572-6e48-47c1-b79b-2277ec9e22b9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.123077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.123675 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.123975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.124256 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.124942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.127128 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.127393 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.128078 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.128358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.128633 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.130741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:12.138269 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1391/2804] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:12.142206 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.157260 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:12.159715 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c671-b0b0-4fb4-88da-5bf52fd117c7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.164532 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.165139 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.165415 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.165680 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.166363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.168298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.179886 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1392/2805] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:12.183942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.195786 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24563f68-6713-407c-bec1-0ffd10d95e31 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.196914 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.198531 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32c6c4-699d-495d-956f-41e7d533292d service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.200413 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.201179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.201463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.201736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] PATH_INFO: `/v3/projects/8f47f3363e66444aa102bd650ffb9eb2/users/34c194e4fb2c40e3949b204fe00f8214/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.208571 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1414/2806] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:34:12.220288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f47f3363e66444aa102bd650ffb9eb2, user_id=34c194e4fb2c40e3949b204fe00f8214, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.223025 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.240926 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1393/2807] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:12 2022] PUT /identity/v3/projects/8f47f3363e66444aa102bd650ffb9eb2/users/34c194e4fb2c40e3949b204fe00f8214/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:12.252893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.253208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.253487 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.283244 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] MFA Rules not processed for user `34c194e4fb2c40e3949b204fe00f8214`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:12.284484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:12.348836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1415/2808] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:12.353437 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.419809 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea46f1a-0998-440a-b948-c935bf10b608 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.426155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.426933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.427223 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.427499 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.428339 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.430520 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.467936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1394/2809] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:34:12 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:12.468204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.481412 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893913-c0ea-4571-91b1-d21990081c10 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.559923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.560700 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.560984 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.561257 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.561940 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.563899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.595157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1416/2810] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:34:12 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:12.598990 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.609578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0d437-d993-41f9-876d-b27869bfa6c8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.615893 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.616578 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.616853 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.617119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.617813 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.619400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.632548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1395/2811] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:12.636848 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.647419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba176a-07bb-4aa9-b046-7c59cc387b60 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.652204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.652612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.652894 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.653171 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] PATH_INFO: `/v3/projects/d2ce2c7024d94696b1ad342a30793e98/users/b61df8ecf5e04eb6bde7adb7a6b27bfe/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.668555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ce2c7024d94696b1ad342a30793e98, user_id=b61df8ecf5e04eb6bde7adb7a6b27bfe, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.671219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.691729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1417/2812] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:12 2022] PUT /identity/v3/projects/d2ce2c7024d94696b1ad342a30793e98/users/b61df8ecf5e04eb6bde7adb7a6b27bfe/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:12.693013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.704565 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c0c9-2965-4696-add6-c154dd59af45 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.771427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.772173 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.772451 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.772725 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.773447 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.775115 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce99448-b8af-40f0-bf0b-529904b74a0e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.780433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:12.787194 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1396/2813] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:12.791299 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.802073 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c57149-d7ec-4812-ba66-e7b6d92d5f26 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.806932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.807594 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.807924 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.808226 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] PATH_INFO: `/v3/projects/d2ce2c7024d94696b1ad342a30793e98/users/b61df8ecf5e04eb6bde7adb7a6b27bfe/roles/abf8a86280c54a0798604e6c88e690cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.810154 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ce2c7024d94696b1ad342a30793e98, user_id=b61df8ecf5e04eb6bde7adb7a6b27bfe, role_id=abf8a86280c54a0798604e6c88e690cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.812825 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.824573 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1418/2814] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:12 2022] PUT /identity/v3/projects/d2ce2c7024d94696b1ad342a30793e98/users/b61df8ecf5e04eb6bde7adb7a6b27bfe/roles/abf8a86280c54a0798604e6c88e690cf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:12.835660 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.835984 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.836241 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.864825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] MFA Rules not processed for user `b61df8ecf5e04eb6bde7adb7a6b27bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:12.866044 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:12.946736 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1397/2815] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:12.958104 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.960206 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:12.973998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:13.063803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b8a5c-9810-4965-aeee-0f8cd00a69ae admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.068282 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.069059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.069349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.069629 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.140130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.142357 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37459e80-5cff-4ecf-9f97-212ca6e17310 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.152623 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1419/2816] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:15.762748 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:15.841467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a28d2-5f7b-402d-8c8e-373305a539f2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:15.847034 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:15.847685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:15.848064 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:15.848548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] PATH_INFO: `/v3/users/34c194e4fb2c40e3949b204fe00f8214` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:15.849768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c194e4fb2c40e3949b204fe00f8214)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:15.851676 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:15.901212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c194e4fb2c40e3949b204fe00f8214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:15.901865 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c194e4fb2c40e3949b204fe00f8214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:15.902467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c194e4fb2c40e3949b204fe00f8214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:15.914871 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c194e4fb2c40e3949b204fe00f8214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:15.921645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c194e4fb2c40e3949b204fe00f8214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:15.934418 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1398/2817] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:15 2022] DELETE /identity/v3/users/34c194e4fb2c40e3949b204fe00f8214 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:15.938998 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:15.939462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:15.939595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:15.959787 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:15.961047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:16.024025 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1420/2818] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:16.035273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:16.036741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:16.044919 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:16.128563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f6285-09cb-494e-9add-8afff4548d20 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:16.132807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:16.133433 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:16.133713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:16.133988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:16.143830 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:16.145312 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:16.154794 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1399/2819] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:16.745558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:16.747495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:16.769462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4efea-f7bb-4fc4-921f-de95deebba7f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:16.777346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:16.778332 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:16.779349 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:16.779859 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] PATH_INFO: `/v3/projects/8f47f3363e66444aa102bd650ffb9eb2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:16.781795 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f47f3363e66444aa102bd650ffb9eb2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:16.784663 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:16.838785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f47f3363e66444aa102bd650ffb9eb2', 'request_id': 'req-4e49565e-1451-4b83-95f3-85efed1551c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:16.843928 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:16.846155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f47f3363e66444aa102bd650ffb9eb2', 'request_id': 'req-4e49565e-1451-4b83-95f3-85efed1551c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:16.855051 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f47f3363e66444aa102bd650ffb9eb2', 'request_id': 'req-4e49565e-1451-4b83-95f3-85efed1551c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:16.867658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] The token cache is being invalidate because project 8f47f3363e66444aa102bd650ffb9eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:16.867999 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:16.868606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:16.870103 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1421/2820] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:16 2022] DELETE /identity/v3/projects/8f47f3363e66444aa102bd650ffb9eb2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:16.874162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:16.883848 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6b1c-01ad-4745-948f-3455cd65f195 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:16.954507 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:16.955243 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:16.955594 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:16.955942 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] PATH_INFO: `/v3/users/b61df8ecf5e04eb6bde7adb7a6b27bfe` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:16.957751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61df8ecf5e04eb6bde7adb7a6b27bfe)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:16.960003 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.009730 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61df8ecf5e04eb6bde7adb7a6b27bfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.010616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61df8ecf5e04eb6bde7adb7a6b27bfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.011506 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61df8ecf5e04eb6bde7adb7a6b27bfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.026361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61df8ecf5e04eb6bde7adb7a6b27bfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.035739 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1512509-c110-4875-aa9b-80bb861c66e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61df8ecf5e04eb6bde7adb7a6b27bfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.051727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1400/2821] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:16 2022] DELETE /identity/v3/users/b61df8ecf5e04eb6bde7adb7a6b27bfe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:17.355581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.357143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:17.370672 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49565e-1451-4b83-95f3-85efed1551c3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.432559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.433200 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.433482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.433757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] PATH_INFO: `/v3/projects/d2ce2c7024d94696b1ad342a30793e98` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.434904 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ce2c7024d94696b1ad342a30793e98)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.436851 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.482814 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ce2c7024d94696b1ad342a30793e98', 'request_id': 'req-820ee112-2793-4e75-9fb3-2f62928c0d80'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.488926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ce2c7024d94696b1ad342a30793e98', 'request_id': 'req-820ee112-2793-4e75-9fb3-2f62928c0d80'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.496304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ce2c7024d94696b1ad342a30793e98', 'request_id': 'req-820ee112-2793-4e75-9fb3-2f62928c0d80'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.508545 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] The token cache is being invalidate because project d2ce2c7024d94696b1ad342a30793e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:17.508836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.509509 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-820ee112-2793-4e75-9fb3-2f62928c0d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:17.511508 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1422/2822] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:17 2022] DELETE /identity/v3/projects/d2ce2c7024d94696b1ad342a30793e98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:17.516514 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.516872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.517219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.538640 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:17.540382 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:17.575834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:17.626521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1401/2823] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:17.635355 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.635724 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.636120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.655642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:17.657388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:17.665293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1423/2824] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:17.669591 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.680318 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e97fb4-6ee3-4d34-9ad1-8b5d4b5cd893 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.684933 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.685537 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.685816 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.686091 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.686774 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:17.687077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.688726 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.700575 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1402/2825] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:17.704804 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.714121 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ae5e7-6a1c-421e-8840-018d31d674fb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.718713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.719335 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.720166 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.720482 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.721143 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.723029 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.749635 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1424/2826] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:34:17 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:17.753782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.764832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a9698-1ad6-4d2b-aa70-25f54ef235df admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.825870 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.826554 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.826826 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.827166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.827836 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.829800 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.856464 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1403/2827] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:34:17 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:17.861620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.875834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b3be-6703-4859-96dd-af7676e4411c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.881893 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.882515 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.882846 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.883556 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.884346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.886068 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.900055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1425/2828] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:17.901272 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.912189 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e02c26-3494-4c01-ac7a-caf99bd77a84 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.917494 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.919440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.919440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.919440 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.924035 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.926212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff3f3f-be4e-4fc2-bf7d-2f3084ae4364 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:17.938432 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1404/2829] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:17.944659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:17.956014 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadd8d0-bcad-4c7a-90b1-33a3690ee65f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:17.960969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:17.961642 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:17.961982 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:17.962197 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] PATH_INFO: `/v3/projects/5da12166801549d1853e1476c1ca403d/users/19140fa9b0544fc3bae0c3409ffabc73/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:17.979741 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da12166801549d1853e1476c1ca403d, user_id=19140fa9b0544fc3bae0c3409ffabc73, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:17.982658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:18.000856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1426/2830] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:17 2022] PUT /identity/v3/projects/5da12166801549d1853e1476c1ca403d/users/19140fa9b0544fc3bae0c3409ffabc73/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:18.011834 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:18.012160 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:18.012445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:18.042116 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] MFA Rules not processed for user `19140fa9b0544fc3bae0c3409ffabc73`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:18.043329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:18.113219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1405/2831] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:18.125271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:18.127203 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:18.143130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:18.194098 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385729a-39e5-43c6-b8f0-391ff372e28c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:18.197314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:18.197936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:18.198218 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:18.198500 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:18.207755 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:18.209227 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:18.218719 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1427/2832] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:18.318750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:18.321047 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:18.349529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:18.392330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f318-7e33-4b11-b433-cfe91a830789 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:18.396651 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:18.397405 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:18.397666 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:18.397950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:18.409831 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:18.484146 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:18.486150 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:18.497950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1406/2833] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:19.168856 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:19.170311 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:19.183382 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08e87-93a4-414e-9227-92d17020890c service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:19.187178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:19.187786 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:19.188117 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:19.188372 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:19.190276 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:19.259173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:19.260645 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:19.270553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1428/2834] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:24.884741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:24.893645 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08335ba8-bef6-487a-a120-80c40dc81530 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.897325 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:24.897936 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:24.898219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:24.898489 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:24.900295 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:24.970569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:24.972765 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:24.982445 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1407/2835] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:26.904969 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.906513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:26.920399 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d815f0-332e-4c8b-a25c-65d3b483c056 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.925469 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.926247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.926622 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.926992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] PATH_INFO: `/v3/users/c1ecb46bae9e4e369416d138203abe6f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.928243 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ecb46bae9e4e369416d138203abe6f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.930473 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.977730 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ecb46bae9e4e369416d138203abe6f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.978388 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ecb46bae9e4e369416d138203abe6f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.979034 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ecb46bae9e4e369416d138203abe6f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.992370 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ecb46bae9e4e369416d138203abe6f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.999460 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ecb46bae9e4e369416d138203abe6f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.012758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1429/2836] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:26 2022] DELETE /identity/v3/users/c1ecb46bae9e4e369416d138203abe6f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:27.220345 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.232974 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28dd7a-d561-4930-989c-a3216bc00e0a service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.330191 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.330857 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.331393 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.331677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] PATH_INFO: `/v3/projects/5084c2b61ad2427d91e0fc85392aee14` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.333427 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5084c2b61ad2427d91e0fc85392aee14)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.335347 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.345271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:27.400945 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5084c2b61ad2427d91e0fc85392aee14', 'request_id': 'req-b5388eb8-6ca6-4382-8c86-26c86e26929e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.411157 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5084c2b61ad2427d91e0fc85392aee14', 'request_id': 'req-b5388eb8-6ca6-4382-8c86-26c86e26929e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.418951 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5084c2b61ad2427d91e0fc85392aee14', 'request_id': 'req-b5388eb8-6ca6-4382-8c86-26c86e26929e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.440615 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] The token cache is being invalidate because project 5084c2b61ad2427d91e0fc85392aee14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:27.441032 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.441812 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.448203 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1408/2837] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:27 2022] DELETE /identity/v3/projects/5084c2b61ad2427d91e0fc85392aee14 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:27.448761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.450431 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.470758 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9422b-47dd-4c23-9c01-835d5e62b462 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.497090 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:27.569794 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.571896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.573164 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.573992 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] PATH_INFO: `/v3/users/f8d0c3bb819441caafac005f5b7463de` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.577247 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d0c3bb819441caafac005f5b7463de)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.581766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.639628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d0c3bb819441caafac005f5b7463de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.644157 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d0c3bb819441caafac005f5b7463de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.645689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d0c3bb819441caafac005f5b7463de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.659503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d0c3bb819441caafac005f5b7463de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.667852 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d0c3bb819441caafac005f5b7463de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.691866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1430/2838] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:27 2022] DELETE /identity/v3/users/f8d0c3bb819441caafac005f5b7463de => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:27.937011 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.938563 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.959528 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5388eb8-6ca6-4382-8c86-26c86e26929e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.048463 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.053187 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.053709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.054002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] PATH_INFO: `/v3/projects/dbc945bd369441218e963648aab071d7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.055687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc945bd369441218e963648aab071d7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.057737 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.121612 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc945bd369441218e963648aab071d7', 'request_id': 'req-83b92397-686a-4059-b282-3f76da0ca3e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:28.127979 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc945bd369441218e963648aab071d7', 'request_id': 'req-83b92397-686a-4059-b282-3f76da0ca3e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:28.135533 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc945bd369441218e963648aab071d7', 'request_id': 'req-83b92397-686a-4059-b282-3f76da0ca3e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:28.156031 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] The token cache is being invalidate because project dbc945bd369441218e963648aab071d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:28.156215 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:28.157738 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83b92397-686a-4059-b282-3f76da0ca3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:28.160239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1409/2839] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:27 2022] DELETE /identity/v3/projects/dbc945bd369441218e963648aab071d7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:37.686891 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:38.153661 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:39.201314 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:39.202833 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:39.215429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9d83-3002-4103-81f4-4c98914e06e2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:39.270267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:39.270905 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:39.271189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:39.271465 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] PATH_INFO: `/v3/users/19140fa9b0544fc3bae0c3409ffabc73` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:39.272907 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19140fa9b0544fc3bae0c3409ffabc73)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:39.274810 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:39.320760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19140fa9b0544fc3bae0c3409ffabc73'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.321426 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19140fa9b0544fc3bae0c3409ffabc73'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.322055 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19140fa9b0544fc3bae0c3409ffabc73'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.334093 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19140fa9b0544fc3bae0c3409ffabc73'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.340781 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19140fa9b0544fc3bae0c3409ffabc73'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.354292 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1431/2840] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:34:39 2022] DELETE /identity/v3/users/19140fa9b0544fc3bae0c3409ffabc73 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:39.358837 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:39.359323 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:39.359436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:39.384531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:39.385741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:39.445616 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1410/2841] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:39.455535 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:39.457100 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:39.513454 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0969f4-f2e9-4176-9dde-f8e52898cb86 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:39.517077 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:39.517704 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:39.517983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:39.518251 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:39.526923 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:39.528548 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3462bef-9403-4af1-8cc0-7d3df11a1766 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:39.538204 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1432/2842] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:39.811569 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:39.813029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:39.825448 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac5c87-9869-4c00-9adb-5945f27ebadf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:39.830687 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:39.831337 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:39.831619 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:39.831889 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] PATH_INFO: `/v3/projects/5da12166801549d1853e1476c1ca403d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:39.833100 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da12166801549d1853e1476c1ca403d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:39.835013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:39.882380 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da12166801549d1853e1476c1ca403d', 'request_id': 'req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.888095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da12166801549d1853e1476c1ca403d', 'request_id': 'req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.894745 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da12166801549d1853e1476c1ca403d', 'request_id': 'req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.906937 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] The token cache is being invalidate because project 5da12166801549d1853e1476c1ca403d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:39.907221 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.907847 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e47f1bfc-5c01-4f63-ac47-569ca27b9ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:39.909959 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1411/2843] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:34:39 2022] DELETE /identity/v3/projects/5da12166801549d1853e1476c1ca403d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:39.914779 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:39.915064 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:39.915336 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:39.934612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:39.935798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:39.951669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:39.995599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1433/2844] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:40.003686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.003999 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.004279 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.024348 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:40.025534 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:40.033364 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1412/2845] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:40.037383 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.046792 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642d319-0407-4343-a6c8-85e857d7ffad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.051439 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.052060 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.052338 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.052606 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.053284 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:40.053582 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.055411 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.067583 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1434/2846] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:40.071810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.082073 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e4541-2970-42b3-b4d2-f9262fa806f0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.086727 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.087327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.087609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.087883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.088639 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.090503 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.129911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1413/2847] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:34:40 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:40.134189 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.144941 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a97cd-bb18-4f4a-a3e1-733ba5f6b752 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.198669 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.199304 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.199590 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.199866 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.200568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.202481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.223727 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1435/2848] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:34:40 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:40.227975 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.238317 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb06b-0673-4bdb-8e7e-05b0586cb802 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.242914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.243531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.243814 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.244126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.244821 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.246438 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.256958 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1414/2849] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:40.261043 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.271612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dfbc1-6e16-4696-9307-b90fabf21a8e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.276519 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.277155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.277437 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.277713 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.278419 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.280050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba7ca5-2a89-46b8-8839-b9290a30f96f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.290628 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1436/2850] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:40.294597 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.304435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9c7c8-916c-4985-ad47-458c8f66465b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.309343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.309997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.310281 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.310559 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] PATH_INFO: `/v3/projects/a013c7af16fb41e8b24409838ef2f388/users/7a8061c380b641788cd3be83385254da/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.325247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a013c7af16fb41e8b24409838ef2f388, user_id=7a8061c380b641788cd3be83385254da, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.328171 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.341443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1415/2851] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:34:40 2022] PUT /identity/v3/projects/a013c7af16fb41e8b24409838ef2f388/users/7a8061c380b641788cd3be83385254da/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:40.405295 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.405514 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.405757 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.433555 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] MFA Rules not processed for user `7a8061c380b641788cd3be83385254da`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:40.434738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:40.501647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1437/2852] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:40.514081 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:40.515811 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:40.538780 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:40.575778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac14b4-e21b-4b2e-aa1f-ebf7574a7849 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:40.579378 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:40.580012 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:40.580338 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:40.580609 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:40.588690 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:40.590092 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:40.598603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1416/2853] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:34:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:50.498705 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:50.594709 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=17 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:03.301884 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:03.303358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:03.360890 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b6af5-9754-489a-9d17-d9d621c8bc13 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:03.365010 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:03.365620 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:03.365899 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:03.366173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:03.367918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:03.438271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:03.440000 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:03.452461 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1438/2854] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:04.638043 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:04.647888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fa334-5c8d-41a0-a0d3-d5187d4c53df service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:04.651435 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:04.652069 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:04.652351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:04.652620 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:04.654371 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:04.669804 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:04.671233 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7392f64-a4c5-49a1-9254-4fc8c4e07fe4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:04.682892 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1417/2855] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:13.447223 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=54 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:14.677768 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:17.722547 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:17.784279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da65ce-2e5a-4359-bfac-879cd41c623b service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:17.789430 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:17.790066 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:17.790346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:17.790640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] PATH_INFO: `/v3/users/7a8061c380b641788cd3be83385254da` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:17.791784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8061c380b641788cd3be83385254da)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:17.793803 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:17.838402 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8061c380b641788cd3be83385254da'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:17.839047 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8061c380b641788cd3be83385254da'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:17.839649 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8061c380b641788cd3be83385254da'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:17.851559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8061c380b641788cd3be83385254da'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:17.858275 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8061c380b641788cd3be83385254da'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:17.870974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1439/2856] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:35:17 2022] DELETE /identity/v3/users/7a8061c380b641788cd3be83385254da => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:17.875521 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:17.876027 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:17.876144 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:17.894607 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:17.896329 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:17.953022 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1418/2857] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:17.963270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:17.964769 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.021063 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20be4-b94c-41e3-95c4-1554598d6b6b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.024671 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.025278 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.025561 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.025836 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.034682 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.036766 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023a946-c96d-49a8-81b3-294ef5c7b528 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.046120 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1440/2858] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:18.318914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.320439 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.333327 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48445d-d9d9-42be-98be-5b718e3874c2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.338807 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.339482 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.339762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.340070 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] PATH_INFO: `/v3/projects/a013c7af16fb41e8b24409838ef2f388` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.341228 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a013c7af16fb41e8b24409838ef2f388)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.343885 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.395214 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a013c7af16fb41e8b24409838ef2f388', 'request_id': 'req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:18.400992 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a013c7af16fb41e8b24409838ef2f388', 'request_id': 'req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:18.407711 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a013c7af16fb41e8b24409838ef2f388', 'request_id': 'req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:18.421454 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] The token cache is being invalidate because project a013c7af16fb41e8b24409838ef2f388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:18.421757 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:18.422365 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-365c7679-2da5-4245-bdb2-b0c4b6aa06a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:18.423872 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1419/2859] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:35:18 2022] DELETE /identity/v3/projects/a013c7af16fb41e8b24409838ef2f388 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:18.429350 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.429640 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.429918 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.450270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:18.451491 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.468607 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:18.512926 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1441/2860] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:18.520601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.520966 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.521302 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.541148 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:18.542335 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.550068 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1420/2861] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:18.554834 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.564619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5646-c480-40cf-a797-d10fb876df36 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.569130 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.569729 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.570008 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.570281 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.570965 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:35:18.571272 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.572885 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.584845 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1442/2862] 10.222.0.72 () {58 vars in 1153 bytes} [Thu Aug 4 10:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:35:18.590664 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.600603 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec770ac-eeb7-45f7-9f2e-26a8b4ef0f5d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.605238 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.605845 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.606126 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.606399 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.607072 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.609041 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.645444 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1421/2863] 10.222.0.72 () {60 vars in 1153 bytes} [Thu Aug 4 10:35:18 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:35:18.649542 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.659784 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468a36a-c05a-4450-b8b1-426c1ecb3df1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.714530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.715133 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.715413 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.715685 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.716376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.718261 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.739530 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1443/2864] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Aug 4 10:35:18 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:18.743486 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.753685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419056-b500-4646-bbbd-3663e8b508d6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.759247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.759858 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.760201 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.760477 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.761196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.762782 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.772585 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1422/2865] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:18.776277 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.786552 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e5a4d-834d-4bee-909b-10affadd691c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.791059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.791656 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.791934 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.792236 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.792922 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.794488 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92806bb5-baa3-49dc-a973-241e3c5a3a8b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.805346 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1444/2866] 10.222.0.72 () {58 vars in 1122 bytes} [Thu Aug 4 10:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:18.809546 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.819907 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281a18-1f8b-4cc0-84c8-7389f60662a0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:18.824431 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:18.825077 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.825353 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.825623 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] PATH_INFO: `/v3/projects/cb4dcadb63b142a89d9aa9d3ce4044d0/users/ec582d8fc1f146faa6af71564edc1f4b/roles/ed22ce3e92574f48b1aaa82fa1077beb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.840218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4dcadb63b142a89d9aa9d3ce4044d0, user_id=ec582d8fc1f146faa6af71564edc1f4b, role_id=ed22ce3e92574f48b1aaa82fa1077beb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:18.842910 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:18.855686 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1423/2867] 10.222.0.72 () {60 vars in 1483 bytes} [Thu Aug 4 10:35:18 2022] PUT /identity/v3/projects/cb4dcadb63b142a89d9aa9d3ce4044d0/users/ec582d8fc1f146faa6af71564edc1f4b/roles/ed22ce3e92574f48b1aaa82fa1077beb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:18.866621 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c3a4-0f0b-47a7-b73d-09a6203b2564 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:18.866909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c3a4-0f0b-47a7-b73d-09a6203b2564 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:18.867178 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c3a4-0f0b-47a7-b73d-09a6203b2564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:18.894362 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f71c3a4-0f0b-47a7-b73d-09a6203b2564 None None] MFA Rules not processed for user `ec582d8fc1f146faa6af71564edc1f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:18.895579 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f71c3a4-0f0b-47a7-b73d-09a6203b2564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.957059 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1445/2868] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:18.967548 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:18.969307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:18.983887 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:19.030296 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a987c-3d53-4e44-ae14-ec84ec180b96 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:19.034020 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a8fe993054ee3a7563058956917d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:19.034635 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:19.034911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:19.035179 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:19.044436 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:19.045911 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20090ee8-5294-48c4-aad6-0170394ab23e service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:19.055253 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1424/2869] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:20.407313 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:20.407983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:20.407983 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:20.427331 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] MFA Rules not processed for user `ec582d8fc1f146faa6af71564edc1f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:20.428559 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:20.435544 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1446/2870] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:29.050641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=21 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:30.421470 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:35.307288 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce720d6-d176-4fa8-b15f-984ed228c156 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:35.307793 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce720d6-d176-4fa8-b15f-984ed228c156 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:35.307901 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce720d6-d176-4fa8-b15f-984ed228c156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:35.355634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ce720d6-d176-4fa8-b15f-984ed228c156 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:35.356891 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ce720d6-d176-4fa8-b15f-984ed228c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:35.420162 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1425/2871] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:35.460248 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:35.461626 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:35.547618 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e62b2-cb05-45f1-b767-dda9e6fc3a82 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:35.553208 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:35.553806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:35.554082 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:35.554351 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:35.562378 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:35.564012 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:35.571788 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1447/2872] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:35:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:35:36.070095 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3998-a8e1-4c51-a2c4-0370a177defd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.070388 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3998-a8e1-4c51-a2c4-0370a177defd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.070670 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3998-a8e1-4c51-a2c4-0370a177defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.090196 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8dfc3998-a8e1-4c51-a2c4-0370a177defd None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:36.091397 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8dfc3998-a8e1-4c51-a2c4-0370a177defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:36.099166 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1426/2873] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:36.111285 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:36.121849 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c153c31-5194-4f63-9dcc-d8399e0982be service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:36.126785 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:36.127380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.127659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.127932 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.135917 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:36.137503 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:36.145944 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1448/2874] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:35:36.361906 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187aec1-0f2f-4b3d-ad37-23fb1f645b8a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.362376 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187aec1-0f2f-4b3d-ad37-23fb1f645b8a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.362492 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187aec1-0f2f-4b3d-ad37-23fb1f645b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.382002 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f187aec1-0f2f-4b3d-ad37-23fb1f645b8a None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:36.383202 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f187aec1-0f2f-4b3d-ad37-23fb1f645b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:36.390255 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1427/2875] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Aug 4 10:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:36.402760 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:36.413513 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160890-7f08-4dca-b44d-66563d0ff399 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:36.418537 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:36.419137 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.419418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.419696 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.427746 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:36.429366 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:36.437634 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1449/2876] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:35:36.755021 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.755307 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.755581 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.773962 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:36.775183 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:36.782443 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1428/2877] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:36.794462 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:36.806358 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf0aa6-409f-4892-bc6b-2f4acd610c36 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:36.812053 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:36.812659 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:36.812942 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:36.813220 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:36.821800 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:36.823376 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498a417-a4f1-4454-940e-5700e6d4f94a service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:36.831293 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1450/2878] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:35:39.243851 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:39.306185 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b785c10-f616-4aba-8eaa-c557b1b0c1ad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:39.310721 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:39.311363 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:39.311641 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:39.311918 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] PATH_INFO: `/v3/users/ec582d8fc1f146faa6af71564edc1f4b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:39.313122 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec582d8fc1f146faa6af71564edc1f4b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:39.315048 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:39.361518 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec582d8fc1f146faa6af71564edc1f4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.362161 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec582d8fc1f146faa6af71564edc1f4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.362756 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec582d8fc1f146faa6af71564edc1f4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.369013 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:39.376935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec582d8fc1f146faa6af71564edc1f4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.383584 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec582d8fc1f146faa6af71564edc1f4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.395883 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1429/2879] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:35:39 2022] DELETE /identity/v3/users/ec582d8fc1f146faa6af71564edc1f4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:39.400558 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:39.401018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:39.401151 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:39.420429 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] MFA Rules not processed for user `bca06f724a4f42d0afd72d25da1e0911`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:39.421636 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:39.479155 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1451/2880] 10.222.0.72 () {58 vars in 958 bytes} [Thu Aug 4 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:39.489531 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:39.490914 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:39.546193 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd055-85ab-4541-990b-86fe770d97d3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:39.549915 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:39.550596 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:39.550876 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:39.551152 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:39.561024 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:39.562446 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:39.571672 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1430/2881] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:39.736883 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:39.737908 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:39.750524 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293acc74-b3dd-41df-ab1a-2ab0effff3b3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:39.755101 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:39.755735 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:39.756050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:39.756322 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] PATH_INFO: `/v3/projects/cb4dcadb63b142a89d9aa9d3ce4044d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:39.757497 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4dcadb63b142a89d9aa9d3ce4044d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:39.759415 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:39.790595 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:39.807568 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4dcadb63b142a89d9aa9d3ce4044d0', 'request_id': 'req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.813004 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4dcadb63b142a89d9aa9d3ce4044d0', 'request_id': 'req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.820738 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4dcadb63b142a89d9aa9d3ce4044d0', 'request_id': 'req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.831666 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] The token cache is being invalidate because project cb4dcadb63b142a89d9aa9d3ce4044d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:39.831980 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.833113 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:39.834586 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1452/2882] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:35:39 2022] DELETE /identity/v3/projects/cb4dcadb63b142a89d9aa9d3ce4044d0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:49.567601 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:49.828808 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:56.228564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:56.230894 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:56.245330 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d691ed-6f71-4468-b0c4-5a562757e301 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:56.304409 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:56.305306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:56.305306 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:56.305629 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:56.307773 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:56.318524 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:56.320542 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b90a9-8a2d-400a-b4af-28f6d454b4e0 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:56.331789 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1431/2883] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:35:57.218377 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:57.219871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:57.322407 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a6a87-12c9-47ff-b38a-4d2d1d6af593 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:57.328958 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45bdd938403d46d6b7b026abd46c564d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:57.331252 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:57.331612 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:57.331974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:57.334391 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:57.348648 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:57.350291 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:57.364270 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1453/2884] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:35:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:03.409377 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:03.409850 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:03.409950 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:03.428761 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] MFA Rules not processed for user `55049003afb04ed4971d769ac2259efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:03.429935 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:03.483219 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1432/2885] 10.222.0.72 () {58 vars in 967 bytes} [Thu Aug 4 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:03.495743 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:03.497146 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:03.577543 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78187-8219-4715-9854-50dfe7732ba4 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:03.582549 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc87ba3c4b9425d91a293211fbc7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:03.583162 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:03.583441 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:03.583709 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:03.591896 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:03.593492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:03.602720 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1454/2886] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Aug 4 10:36:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:36:06.998090 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.999628 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.078634 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17c0b-1c1d-4496-9848-a60933db0f55 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:07.083767 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:07.084467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:07.084691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:07.084960 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:07.086685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.098779 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:07.100972 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:07.111386 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1433/2887] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:10.198906 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.200495 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:10.277667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b477c-da4c-446a-a031-f51cbf004659 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.282271 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.282909 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.283195 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.283472 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] PATH_INFO: `/v3/users/edd0f098e40c4c2da8d55df9a8fa0912` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.284702 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd0f098e40c4c2da8d55df9a8fa0912)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.286619 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.304863 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:10.337647 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd0f098e40c4c2da8d55df9a8fa0912'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.338348 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd0f098e40c4c2da8d55df9a8fa0912'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.338956 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd0f098e40c4c2da8d55df9a8fa0912'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.351187 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd0f098e40c4c2da8d55df9a8fa0912'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.357989 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd0f098e40c4c2da8d55df9a8fa0912'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.370581 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1455/2888] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:36:10 2022] DELETE /identity/v3/users/edd0f098e40c4c2da8d55df9a8fa0912 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:10.554008 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.564625 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b5a6c-3da6-4212-aed9-f2ee314842b1 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.600298 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:10.626119 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.626685 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.626961 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.627247 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] PATH_INFO: `/v3/projects/f63d8ac72593445089042fd7c3a31651` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.628424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63d8ac72593445089042fd7c3a31651)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.630316 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.677996 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63d8ac72593445089042fd7c3a31651', 'request_id': 'req-1388c9ae-a32a-41ba-938c-30d6009fec8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.684212 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63d8ac72593445089042fd7c3a31651', 'request_id': 'req-1388c9ae-a32a-41ba-938c-30d6009fec8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.690218 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63d8ac72593445089042fd7c3a31651', 'request_id': 'req-1388c9ae-a32a-41ba-938c-30d6009fec8d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.703400 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] The token cache is being invalidate because project f63d8ac72593445089042fd7c3a31651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:10.703691 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.704343 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.705895 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1434/2889] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:36:10 2022] DELETE /identity/v3/projects/f63d8ac72593445089042fd7c3a31651 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:10.710144 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.711553 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:10.725368 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfc5a4-d449-4733-85f6-ee2888feeb43 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.780083 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.780710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.780991 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.781263 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] PATH_INFO: `/v3/users/81504d3f2d9741bea68401206237c274` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.782406 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81504d3f2d9741bea68401206237c274)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.785175 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.831044 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81504d3f2d9741bea68401206237c274'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.831761 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81504d3f2d9741bea68401206237c274'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.832418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81504d3f2d9741bea68401206237c274'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.844484 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81504d3f2d9741bea68401206237c274'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.851667 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81504d3f2d9741bea68401206237c274'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:10.863798 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1456/2890] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:36:10 2022] DELETE /identity/v3/users/81504d3f2d9741bea68401206237c274 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:11.035332 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:11.036810 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:11.050033 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388c9ae-a32a-41ba-938c-30d6009fec8d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:11.105825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:11.106467 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:11.106750 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:11.107029 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] PATH_INFO: `/v3/projects/a1cdd618739b4319bbaf7a3258f6937f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:11.108235 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cdd618739b4319bbaf7a3258f6937f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:11.110143 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:11.157677 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cdd618739b4319bbaf7a3258f6937f', 'request_id': 'req-645d9976-8d85-4323-91f8-c4b98900d3c4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:11.163513 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cdd618739b4319bbaf7a3258f6937f', 'request_id': 'req-645d9976-8d85-4323-91f8-c4b98900d3c4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:11.169490 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cdd618739b4319bbaf7a3258f6937f', 'request_id': 'req-645d9976-8d85-4323-91f8-c4b98900d3c4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:11.177555 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:11.182957 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] The token cache is being invalidate because project a1cdd618739b4319bbaf7a3258f6937f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:11.183355 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:11.184237 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:11.186564 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1435/2891] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:36:11 2022] DELETE /identity/v3/projects/a1cdd618739b4319bbaf7a3258f6937f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:20.860425 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=19,keystone:SELECT=73,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:21.188896 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:25.900225 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:25.901728 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:25.958806 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca5b81-0b13-4e46-a1bd-386400058d48 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:26.008773 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd48c3743968a4f3da87a246e8f5f6e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:26.009381 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:26.009661 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:26.009949 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:26.011710 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:26.072837 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:26.074267 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:26.083812 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1457/2892] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:32.088740 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:32.090424 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:32.105094 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d9976-8d85-4323-91f8-c4b98900d3c4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:32.109825 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:32.110466 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.110749 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.111052 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] PATH_INFO: `/v3/users/fd829e78e710440687678701fea3cf2c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.112322 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd829e78e710440687678701fea3cf2c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:32.114239 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:32.169485 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd829e78e710440687678701fea3cf2c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.170130 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd829e78e710440687678701fea3cf2c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.170762 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd829e78e710440687678701fea3cf2c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.185050 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd829e78e710440687678701fea3cf2c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.192689 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd829e78e710440687678701fea3cf2c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.205702 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1436/2893] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:36:32 2022] DELETE /identity/v3/users/fd829e78e710440687678701fea3cf2c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:32.383652 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:32.393871 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b904703-a334-4b0a-9b32-a04d8a28d3bc service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:32.453279 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:32.453936 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.454219 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.454492 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] PATH_INFO: `/v3/projects/82b077dd80c54b08ba3ad79b8cafd3f7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.455658 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b077dd80c54b08ba3ad79b8cafd3f7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:32.457599 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:32.465585 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:32.504748 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b077dd80c54b08ba3ad79b8cafd3f7', 'request_id': 'req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.510096 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b077dd80c54b08ba3ad79b8cafd3f7', 'request_id': 'req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.517418 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b077dd80c54b08ba3ad79b8cafd3f7', 'request_id': 'req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.528049 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] The token cache is being invalidate because project 82b077dd80c54b08ba3ad79b8cafd3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:32.528328 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.528925 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.530401 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1458/2894] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:36:32 2022] DELETE /identity/v3/projects/82b077dd80c54b08ba3ad79b8cafd3f7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:32.534373 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:32.535751 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:32.549204 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563e82-f031-467a-9f90-8772a898361b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:32.604273 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:32.604832 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.605113 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.605383 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] PATH_INFO: `/v3/users/d20f5320adee461e9e90bc4465186913` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.606549 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20f5320adee461e9e90bc4465186913)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:32.608480 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:32.654997 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20f5320adee461e9e90bc4465186913'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.655646 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20f5320adee461e9e90bc4465186913'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.656270 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20f5320adee461e9e90bc4465186913'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.668271 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20f5320adee461e9e90bc4465186913'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.674988 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20f5320adee461e9e90bc4465186913'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.676644 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:32.690778 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1437/2895] 10.222.0.72 () {58 vars in 1224 bytes} [Thu Aug 4 10:36:32 2022] DELETE /identity/v3/users/d20f5320adee461e9e90bc4465186913 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:32.862916 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:32.864481 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:32.878050 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fbf29-1b85-4b6e-9b3f-0fc8f1272bb5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:32.934046 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca06f724a4f42d0afd72d25da1e0911', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d75901b5a184df7b4b0c986ca5abc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:32.934689 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.934972 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.935250 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] PATH_INFO: `/v3/projects/bb80a63667074f8f94417ad29b54bda3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.936432 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb80a63667074f8f94417ad29b54bda3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:32.938339 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:32.982974 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb80a63667074f8f94417ad29b54bda3', 'request_id': 'req-b83e534d-5443-4556-9ed2-497d12dda9c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.988380 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb80a63667074f8f94417ad29b54bda3', 'request_id': 'req-b83e534d-5443-4556-9ed2-497d12dda9c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:32.996173 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb80a63667074f8f94417ad29b54bda3', 'request_id': 'req-b83e534d-5443-4556-9ed2-497d12dda9c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:33.007608 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] The token cache is being invalidate because project bb80a63667074f8f94417ad29b54bda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:33.007888 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:33.008536 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b83e534d-5443-4556-9ed2-497d12dda9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:33.010107 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1459/2896] 10.222.0.72 () {58 vars in 1233 bytes} [Thu Aug 4 10:36:32 2022] DELETE /identity/v3/projects/bb80a63667074f8f94417ad29b54bda3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:42.687888 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:43.004018 dv-cinder-826608-8-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:19.332351 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:19.333741 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:19.401529 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a22f-33ba-4665-b054-dc70f5fe631f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:19.463088 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47da679abbac42a1b3244ff9acafd577', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '212cd92bbff94ea9af0e07b03238b9ad', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:19.463713 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:19.464016 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:19.464290 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:19.466605 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:19.481361 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:19.483007 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5382e-213c-4359-8b0a-a6ec2476ed29 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:19.494412 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1438/2897] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Aug 4 10:38:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:29.489155 dv-cinder-826608-8-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=58 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}